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Introduction 



This monograph will be the fourth MDC publication since 1976 offering an 
objective comparison of commercial vocational evaluation systems. Beginning 
in 1976 with A Comparison of Seven Vocational Evaluation Systems , continuing 
with A Comparison of Four Vocational Evaluation Systems in 1977 and, most 
recently, the 1980 A Comparison of Commercial Vocational Evaluation Systems , 
MDC has sought to provide vocational evaluators, special education teachers, 
vocational educators, manpower trainees, corrections personnel and program 
administrators in many human service fields with accurate and detailed infor- 
mation on widely available commercial vocational evaluation systems. 

While it has only been two years since the 1980 Comparisons publication, 
this revised edition is intended to reflect major changes both inside and out- 
side the industry during this time period. Within the evaluation system 
industry there have been two significant changes. The first is the develop- 
ment of three new systems: Occupational Assessment/Evaluation Systems (OA/ES), 
the System for Assessment and Group Evaluation (SAGE), and the Work Skill 
Development Package (WSD). The first two of these three new systems are de- 
signed to give a complete aptitude assessment in a short period of time. At 
the other end of the spectrum is the WSD; this is intended to be a training 
and evaluation tool and is designed for teaching basic skills over a compara- 
tively long time span. The second innovation has been major changes in the 
"software" of several systems. New manuals have been written for the TAP and 
Career Evaluation System (i.e., Hester). A computerized report format is now 
available for the McCarron-Dial and the Career Evaluation System printout has 
been revised. 

Some major changes have occurred outside of the industry in the last few 
years which have changed the use of vocational evaluation systems dramatically. 
First, due mostly to reductions in rehabilitation funding, many state voca- 
tional rehabilitation agencies have reduced the amount of time allowed for 
vocational evaluation from two to three weeks to one week or less. Evaluators 
must cope with the problem of obtaining accurate information about clients in 
shorter time periods. One solution to this problem is to use tools which give 
large amounts of data in a short period of time. Isolated trait work samples, 
simulations, and psychological testing offer potential solutions to these 
problems. Second, also due to funding reductions, the strategy of many re- 
habilitation professionals has changed from one of a lengthy vocational 
evaluation, usually followed by work adjustment and skill training prior to 
placement to one of direct job placement after a short evaluation period. 
This forces the evaluator to deal less with the clients ultimate potential 
and more with direct transfer of skills and aptitude to currently available 
jobs. 

Third, and happening over a longer period of time, has been the contin- 
ued use of vocational evaluation method? and techniques into areas outside of 
the state-federal vocational rehabilitation system. The expansion into edu- 
cation programs in particular is creating an entire new area which will 
challenge the skills and talents of the teacher and the evaluator. Finally, 
there has been the significant increase in the number of people involved in 
private for profit rehabilitation. Whether dealing with litigation, case 
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management, training or placement, these companies need accurate, fast meth- 
ods of assessing client potential. 



Based on changes within the evaluation system industry and changes out- 
side that industry, there are several current trends that most likely will 
become stronger in the future. For both nonprofit and profit vocational re- 
habilitation programs, there will be a continued emphasis on shorter time 
periods for vocational evaluation. Indeed, as this occurs, professionals will 
have to distinguish between a traditional evaluation model and a newer assess- 
ment model. The advent of inexpensive computer technology over the past few 
years has already resulted in three evaluation systems (i.e., Career Evalu- 
ation System, SAGE, OA/ES) that have person- job matching systems based more or 
less on the Dictionary of Occupational Titles . This trend will most certainly 
continue as evaluators find that many of the tedious tasks of searching can be 
given to a machine. Finally, there is the possibility the job sample and sim- 
ulated work samples that have been the standard for about 15 years will be to 
a degree replaced by isolated trait work samples and psychometric instruments. 

There are two problems that vocational evaluators must face with regard 
to commercial evaluation systems. The first is that to some people, both in 
and out of the profession, the term "vocational evaluation 11 has come to mean 
"work samples" in general and "commercial work samples" in particular. While 
this is a problem that can only be solved by evaluators themselves, it must 
be stressed to the readers of this publication that there are other methods 
of assessing persons besides commercial work samples. The second problem is 
one of technical standards. While some development of norms, reliability, 
and validity have occurred in the last 14 years, most work sample systems 
are technically inadequate. The evaluator who would carefully read the ad- 
ministration and technical manuals before selecting a new test would often be 
the same person who would select a work sample system based on "face" validity 
and on "norms" developed on limited or nonexistent populations, While the 
developers of these systems are at fault, it is the evaluator who is ultimately 
responsible to his clients for the selection of inadequate assessment tools. 

In preparing this comparison, manuals, technical reports, and related 
publications were used to obtain information about each system. It is hoped 
that this publication will be used as a guide for potential purchasers so 
that they can examine each system in light of their own needs. Facilities 
considering the purchase of any system should not only talk with vocational 
evaluators in facilities who are using a system, but should also see the sys- 
tem in action prior to making a final decision. 

This publication contains four sections. The first is an edited reprint 
of an article on how to select a commercial vocational evaluation system 
(Botterbusch and Sax, 1977); this article is based on the introduction to the 
earlier comparison publications. The second is an explanation of the 15 major 
points contained in the outline. The third section is a table which presents 
a very brief comparison of the systems on the first ten points in the outline 
(points 12 through 15 are not presented because of redundancy or not being 
appropriate to summarize) . The fourth section contains a more detailed de- 
scription of each vocational evaluation system, including reviewers comments, 
address, cost, and references. 
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Two additional comments are necessary. First, because most systems are 
constantly being revised, expanded,, and updated, the potential user should 
contact the manufacturers for the most recent information. Second, for those 
who desire additional information, the Materials Development Center has a 
sound/slide presentation on most of the vocational evaluation systems described 
in this publication. (These interested in the sound/slide presentations should 
write MDC for a brochure describing this series.) 

I would like to thank Ms. Arlyn Treadwell who typed this document. Fi- 
nally, I would also like to thank all of the commercial developers who have 
willingly provided manuals, technical reports, forms, etc. 



Karl F. Botterbusch, Ph.D. 
October, 1982 
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Some Considerations for the Selection of a Commercial 
Vocational Evaluation System 



Over the years many people have contacted the MDC for advice concerning 
the most appropriate vocational evaluation system to purchase for their facil- 
ity. This section was prepared in response to these requests and will outline 
some of the factors to be considered prior to purchasing a commercial voca- 
tional evaluation system. The evaluator has at his disposal many tools for 
assessing client potential (Task Force No. 2, 1975), These tools fall within 
one of the following four categories: 

1. On-the-Job Evaluations - These are situations in which the client is 
assessed in one or more of a variety of real work situations includ- 
ing: job site situations in industry, trial training evaluation in 
a training program, and simulated job stations within the facility. 

2. Sheltered Employment - This offers the evaluator an opportunity for 
assessing the client under working conditions that should be similar 
to those found in competitive employment. 

3. Work Samples - There are four types of work samples according to 
their degree of correspondence with actual jobs: actual job samples, 
simulated job samples, cluster trait samples, and single trait sam- 
ples. 

4. Psychological Tests - These include an almost endless variety of 
paper-and-pencil and apparatus techniques for measuring traits, 
abilities, and related characteristics of an individual. 

Faced with the need to equip and administer a vocational evaluation unit, many 
untrained and inexperienced evaluators feel that the purchase of a commercial 
evaluation battery will solve their problems. The evaluator should analyze a 
number of factors in deciding how to equip the evaluation unit and then care- 
fully investigate all the tools listed in the above categories to determine 
the ones that will provide him with the best methods to adequately assess his 
clients, 

The first area of consideration is the relationship between the community 
and the vocational evaluation unit. The evaluator must carefully investigate 
the range and type of jobs that are available in the local labor market. Thus, 
a small rural facility or a facility in a one industry community will have a 
narrower range of job evaluation stations than a facility in an urban area. 
Labor market information can be obtained through vocational surveys, local em- 
ployment offices and agencies, and client placement records. Once potential 
employment opportunities have been determined, intelligent decisions can be 
made on what type of evaluation tools can best assess these demands. 

Because the evaluation outcomes may not result in immediate placement, it 
is also necessary to investigate the training opportunities available for cli- 
ents and these should also be reflected in the selection of evaluation tools. 
A client's range of occupations widens and his chances for upward mobility are 
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frequently increased as a result of training. The presence of an area voca- 
tional-technical school, private trade and business schools, on-the-job 
training programs, apprenticeship programs, and even higher education should 
be reflected in the evaluation unit. Vocational evaluation techniques cover- 
ing a wide variety of occupational areas and assessing the full range of 
client aptitudes and interests are needed if the facility is in an area where 
many employment and training opportunities are available. 

The second consideration is the client population. Some evaluation units 
must be capable of serving clients with all types of mental, physical, psycho- 
logical, and cultural disabilities. Other facilities restrict themselves to 
serving either a single disability or a small number of disabilities. A fa- 
cility dealing with many types of handicaps would generally need to have 
techniques covering the entire range of occupational areas and skill levels 
within these areas. A facility providing services to a single disability 
group could safely limit its evaluation areas. For example, a facility serv- 
ing only mentally retarded clients could realistically avoid evaluation for 
occupations that require a great deal of formalized training or higher educa- 
tion. Some systems claim to have been designed specifically for a particular 
level of client functioning. When selecting evaluation tools, keep in mind 
the type of clients served since it would be a waste of time to assess a 
client for a job he could not fil 1 because of his handicap. At the present 
time, most commercial vocational evaluation systems are designed "for persons 
who can see and hear and most contain no special instructions or modifications 
for the blind or deaf. The evaluator should be aware that he frequently will 
have to make modifications in commercial work samples so that they meet the 
special needs of his clients (Botterbusch, 1976; Dickson, 1976). In summary, 
if an evaluator is considering a commercial evaluation battery, he/she should 
check the battery against the needs of the client population served and then 
decide: (1) whether the system is designed for the target disability group(s), 
or (2) whether other evaluation techniques would be more appropriate. 

The third area to be considered is the purpose of evaluation. Although 
all vocational evaluation techniques should provide career information, a 
particular technique may either emphasize occupational information by pro- 
viding a hands-on experience or it may emphasize the assessment of present 
skills and aptitudes without relating it to career information. Some systems 
attempt to provide a thorough evaluation of the client's aptitudes and work 
behaviors; others provide occupational information and experience, often at 
the expense of a thorough ability assessment. The evaluator should check 
the final report format to determine exactly what information it contains; 
this goes a long way in determining the purpose of a particular system. The 
evaluator must first decide what needs to be included in these areas and then 
attempt to find or develop the evaluation tools that best fit the client's 
needs. A system should never be purchased to "fit in somewhere." 

The fourth area of concern is perhaps the most basic— why even purchase 
a commercial evaluation system at all? All of the systems are relatively 
expensive; some are very expensive. None will probably meet the individual 
needs of a facility in terms of community jobs and training, client popu- 
lations, and purpose of evaluation. A facility could develop its own eval- 
uation unit based on job or work samples taken from local industry. This 
would make evaluation very realistic for the client, staff, and even for a 
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potential employer. Additional work samples could be developed from existing 
subcontracts in the workshop. This method not only will assess the areas in 
which the client has his maximum functions and interests, but also the areas 
of the shop that would best fit the client. In addition, the client would 
receive training on the work performed in the workshop. Then, when the cli- 
ent is transferred from the evaluation unit, he or she will be familiar with 
the subcontract, which should alleviate the need for the supervisor to train 
the client from "scratch." 

The development of a work sample is expensive in terms of staff time. 
In most facilities, staff are hired to provide direct client service and to 
have a staff person doing developmental work reduces the time available for 
working with clients. Few evaluation units can afford the luxury of develop- 
mental time for staff persons. Besides the time element, development of 
evaluation tools demands a working knowledge of the skills required to per- 
form jobs and to analyze tasks, of form and report design, of behavior analy- 
sis, of statistics for norms, and of industrial engineering techniques. 
Although these skills are becoming more and more widespread among evaluators, 
there are still many facilities that lack persons with these competencies. 
The lack of developmental time coupled with the inexperience of some evalu- 
ators is partly responsible for the increased use of commercial evaluation 
systems. The purchase of these systems as a matter of convenience does not 
necessarily imply that the systems are not useful to the evaluator. 

The first decision is whether the evaluation unit is meeting client needs 
in terms of accurate assessment for available jobs and/or training. If needs 
are not being met, the second decision becomes a question of what areas of job 
assessment are needed for the evaluation unit,, After these needs are known, 
a thorough review of the different evaluation techniques, commercial voca- 
tional evaluation batteries (or parts of these batteries), and other available 
resources should be made to determine how to best meet these needs. However, 
it is a common practice for many persons to want to buy a system that will 
give all the answers. Such a system simply does not exist. There is also the 
possibility of carefully selecting individual work samples from several sys- 
tems and combining these into a unified system specific to the needs of the 
facility. To have appropriate evalualtion stations, there has to be a great 
deal of analysis of what is to be accomplished during evaluation, the avail- 
able jobs and training opportunities, the types of clients with whom you are 
working, and the best way to accomplish the goals of your facility. This 
analysis is absolutely necessary before a workable system can be developed. 

The preceding points should only be used as general guidelines because 
each facility is unique. A critical factor in purchasing a system should be 
based on the knowledge of what is needed and not on the cost or attractive- 
ness of the hardware. Usually, no one system will meet all the needs of a 
facility and the purchased system should be integrated with facility con- 
structed devices, other evaluation systems, on-the-job evaluation, and psy- 
chological tests. MDC suggests that a facility obtain as much accurate 
information as possible about a system prior to purchase. Some sources of 
information are: 

1. The information contained in this rresent publication. 
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2. MDC's sound/slide presentations on most evaluation systems. 
A brochure listing these programs is available from MDC. 

3. Talk with the evaluators in other facilities who are using 
the system being considered and see what they think cf it. 

4. If possible, try out the system yourself with clients in 
another facility. 

5. Write the system's manufacturer and obtain current informa- 
tion. 

In conclusion, MDC cannot recommend what commercial system(s) will be best for 
a facility because selecting the appropriate system is (or should be) based 
on an accurate, realistic assessment of the unique needs of each facility. 
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Vocational Evaluation System Outline 



Development 

a. Sponsor - The organization that originally funded or financed the 
development of the vocational evaluation system, 

b. Target Group - What specific populations, such as disadvantaged, 
mentally retarded, or physically handicapped, was the system 
designed to serve? 

c. Basis l. % the System - What theoretical or organizational principle, 
such as the Dictionary of Occupational Titles , was used as a basis 
for development? 

Organization 

a. Name and Number of Work Samples - How many work samples does the 
system contain? What are the names of the work samples? 

b. Grouping of Work Samples - What is the arrangement of the individual 
work samples within the system? Are several work samples grouped 

in a hierarchy or is each work sample independent? 

c. Manual - What are the organization and contents of the manual (s)? 
Does it provide all the details that the evaluator needs to know 
in order to use the system? 

Physical Aspects 

a. Packaging of the Work Samples - How are work samples packaged for 
sale? Does each work sample "stand alone" or must tools and equip- 
ment be shared with other work samples? 

b. Durability - How durable are the tools and equipment in the system? 
If the system uses audiovisual components, how prone to breakdown 
are they? 

c. Expendable Supplies - How much and what type of expendable supplies 
(e.g., wood, paper, wire) are needed per client? 

d. Replacement - To what degree can supplies and materials (e.g., tools, 
nuts and bolts, colored chips) be obtained locally or must they be 
ordered from the developer? 

Work Evaluation Process 

a. Preliminary Screening - What information is needed or what decisions 
must be made before a client can be administered the system? 

b. Sequence of Work Sample Administration - In what order are the work 
samples administered? 
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c. Client Involvement - To what extent is the client informed of his/h 
progress during the course of administration? What type, if any, o 
formal feedback is given to the client after the entire battery has 
been administered? What type of contact does the client have with 
the evaluator? 

d. Evaluation Setting - Does the general environment attempt to Simula 
industry, produce a classroom atmosphere, or resemble a formal test 
ing situation? 

e. Time to Complete the Entire System - How long does it take the aver 
age client to complete all the work samples in the system? 

Administration 

a. Procedures - Are the purposes of each work sample, materials needed 
layout, and general instructions clearly given so that there is 
little chance of misinterpretation? 

b. Method of Instruction Giving - How does the client receive his/her 
instructions for the work samples in the system? For example: ora' 
demonstration, written instructions, or audiovisual? 

c. Separation of Learning/Performance - Does the work sample have sepa- 
rate practice (learning) and performance periods? Are there definil 
criteria (e.g., three correct assemblies; the lines drawn within t 
1/16 inch) that must be met before the client can progress from a 
practice period to a performance period. 

d. Providing Assistance to the Client - What procedures are there for 
giving extra or additional instructions, demonstrations or feedback 
after the period of initial instructions? 

e. Repeating Work Samples - What provisions are made for the re-adminis 
tration of some work samples and what is the purpose of re-admin- 
istration? 

Scoring and Norms 

a. Timing - What are the procedures for timing the client? 

b. Timing Interval - When does the evaluator start timing the client an 
when does he stop? Are there specific cut-offs or does the client 
continue until the work sample is completed? 

c. Time Norms - What is the procedure for reporting the time score for 
each work sample? 

d. Error Scoring - What procedures, such as a random check of some part 
general rating of overall quality, or a comparison to standards, are 
used for determining errors? 

e. Scoring Aids - What use is made of overlays, templates, models, etc. 
to make scoring more accurate and easier for the evaluator? 



f. Quality Norm? - What procedures are used for reporting the number of 
errors, quality ratings, etc., for each work sample? What, if any, 
type of a rating system is used? 

g. Emphasis in Scoring - Does the system emphasize time or errors in 
the scoring process or are both given equal weight? 

Observation of Clients 

a. Work Performance - Are work performance factors (e.g., fine finger 
dexterity, color perception) listed for the system and are specific 
work performance factors given for each work sample? 

b. Work Behaviors - Are work behaviors (e.g., ability to follow instruc- 
tions, communication with supervisors) defined for the system and are 
specific work behaviors to be observed for each work sample? 

c. Recording System - What procedures does the system have for the re- 
cording, describing, and rating of observed work performance and 
work behaviors? 

d. Frequency of Observation - How often and to what extent is the eval- 
uator to observe and record client behavior? 

Reporting 

a. Forms - What forms for recording time and quality, work performance, 
work behavior, etc., are used for each work sample in the system? 

b. Final Report Format - What information is included in the final re- 
port and what type of format (<>.g., rating scales, free narration) 
is used to present the information? Is a final report format and/ 
or example given in the work sample manual? 

Utility 

a. Vocational Exploration - Does the system provide experiences that 
the client can readily rebate to real jobs? 

b. Vocational Recommendations - Are training and job recommendations 
specific or general? How are they related to the DOT or other 
job classification systems? Can extended evaluation work adjust- 
ment, etc., be recommended as a result of this system? 

c. Counselor Utilization - Can the system provide the counsel or or 
referring agency with useful information and to what extent is the 
counselor involved in the process? 

Training in the System 

a. Training Required - Is formal training required before the system 
is sold? 

b. Training Available - Is formal training available? Where is it 
available? 



c. Duration - How much time is required for training? 

d. Follow-up - Is technical assistance available after purchase and 
training? 

Technical Considerations 

a. Norm Base - On what types of populations (e.g., client, employed 
workers, general populations) was the system normed, and are these 
norm groups clearly defined? Are norm groups of adequate size 
for practical use? Are predetermined time standards, such as 
Methods-Time-Measurement, used? 

b. Reliability - What empirical evidence is there to demonstrate that 
the system and its component work samples gives reliable or con- 
sistent results? Are the research methods, sample sizes, etc., 
described in enough detail to permit the user to judge the meaning- 
fulness of any data? 

c. Validity - What content, construct or empirical validity data is 
available to indicate that the system really does what it claims, 
such as make more realistic choices, job and/or training success, 
etc.? Are research methods, sample sizes, etc., described in 
enough detail to permit the user to judge the meaningful ness of 
any data? 

Reviewer's Summary and Consents - This contains what the reviewer con- 
STQt.."S to be the major advantages and disadvantages of the system. Also 
included are any unique points about each system and some ideas for its 
use. 

Address - The address of the manufacturer is given for those wishing to 
obtain additional information. 

Cost - The present cost of the system and what materials and services 
are included in the price. 

References - All generally available references are given; those not 
available from the MDC Loan Service are indicated by an asterisk (*) . 
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Outl ine 


Career Hester 


McCarron-Dial 


Micro-TOWER 


1. Development 

a. Sponsor 

b. Target Group 

c. Basis of System 


Edward Hester 

all intelligence levels, 
physically disabled 
normal populations 

DOT 


McCarron & Dial 

mentally retarded, mentally 
ill, learning disabilities 

3 Mtrurupbycno i ogi ca i Tac- 
tors 


ICD Rehabilitation & 
Research Center 

general rehabilitation 
population 

DOT 


2. Organization 

a. Number of Work 
Samples 

b. Grouping of 
Work Samples 

c. Manual 


39 test scores 
each independent 

contains all system 
details; computer 
printed 


17 

grouped into 5 factors 
4 manuals; very detailed 


13 

5 groups of general 
aptitudes 

general manual, separate 
manual for each work 
sample contains all system 
details 


3. Physical Aspects 

a. Packaging of 
Work Samples 

b. Durability 

c. Expendable 
Suppl ies 

d. Replacement 


some individually 
packaged 

estimate fairly durable 
staples and paper 

SUDDlieS locallv. Darts 
from distributor 


5 separate briefcase-like 
kits 

not appl icable 

no consumable materials 
used 

mus t be ordered from man u- 
facturer 


each work sample packaged 
separately 

durable 

wire only 

al 1 forms local ly if 
desired 


4. Work Evaluation 
Process 

a. Preliminary 
Screening 

b. Sequence of 
Administration 

c. Client 
Invol vement 

d. Evaluation 
Setting 

e. Time to Complete 
Entire System 


not required 
no specified order 
little during testing 
formal testing setting 
5 hours 


client interview 

in order by factors 

encouraged 

formal testing and 
workshop 

2 weeks recommended 


not required 

discretion of evaluator 

extensive client involvement 

combination of formal test- 
ing and counseling 

15-20 hours 
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OA/ES 


JEVS 


Prep 


Valpar #17 


Individualized Rehabilita- 
tion Proarams 


U.S. Department of Labor 


Prep, Inc. 


Valpar International 


industrially disabled 


initially for disadvant- 
aged 


manpower, secondary educa- 
tion, and special needs 


mentally retarded 


tions 


DOT 


15 career systems of USOE 


not specified 


11 tests; 9 work samples 


28 


27 


11 assessment techniques 
using different formats 


by aptitude, interest, physi- 
cal capacity scale and work 
sample 


12 Work Groups 


each is independent 


5 areas 


3 manuals, omits many 
system details 


contains all system 
details 


general manual and separate 
manual for each work sample, 
contains all details 


general manual ; separate 
manual for each area; 
contains all details 


packaged as a group 


each work sample 
packaged separately 


each separately packaged 
in a portable container 


each of the 5 areas 
packaged separately 


estimate durable 


very durable 


durable 


very durable 


forms, wire and solder 


paper* fabric, string 


wood, sheet metal, wire, 
etc. 


no consumable materials 


LC3 L3 1 1 Ulll pUU 1 1 JilC 1 3 , 

parts locally 


most purchased locally 


suppl ies local ly 


forms ordered from developer 
or locally reproduced 


not required 


not required 


not required 


not specified 


no specified order 


progressive from least 
to most complex 


any order 


any order 


little during testing 


some 


extensive client 
invol vement 


considerable 


formal testing setting 


realistic work setting 
stressed 


classroom atmosphere 


not speci fied 


4 hours without work 
samples 


6-7 days 


average - 2 hours per 
work sample 


SH hours 
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Outl ine 


Career Hester 


McCarron-Dial 


Micro- TOWER 


5. Administration 

a. Procedures 

b. Method of In- 
struction Giving 

c. Separation of 
Learning/Perfor- 
mance 

d. Providing Assis- 
tance to Client 

e. Repeating Work 
Samples 


specified in detail 

oral and demonstra- 
tion 

not appl icable 

no assistance after 
timing begins 

if necessary, after 
two weeks 


specified in detail 
oral and demonstration 

not applicable 

little assistance provided 
if necessary 


specified in detail 

audio cassette, evaluator 
demonstrations 

stressed, almost total 

no assistance after 
timing begins 

not specified 


6. Scoring and Norms 

a. Timing 

b. Timing Interval 

c. Time Norms 

d. Error Scoring 

e. Scoring Aids 

f. Quality Norms 

0. Emphasis in 
Scoring 


evaluator times client 

varies with type of test 

no spearate time norms 
gi ven 

no separate error scores 
given 

some use 

not used in testing 

time to completion or 
number of responses 


evaluator times client 
specified time limits 
some separate time norms 
compared to standards 
not used 

combined with time norms 
for overall score 

qual ity 


cassette tape 

specified time for each 
work sample 

no time norms used 

number completed; pieces 
correct 

some use 

rated on 5 point scale 
emphasis on quality 


7. Observation of 
Clients 

a. Work Performance 

b. Work Behaviors 

c. Recording 
System 

d. Frequency of 
Observation 


Because the Career 
Hester uses psychological 
and psychophysical tests, 
> no behavior observations 
are made, except for es- 
timates on the people 
functions 


factors identified 

clearly defined 

2 separate instruments used 

2 hours for 5 days 


no specific factors de- 
fined 

5 work behaviors listed 
none 

frequent observations 
expected 


8. Reporting 

a. Forms 

b. Final Report 
Format 


standardized forms for 
all phases 

computer generated re* 
Dort lists SDprifir 

pwi u 1 1 w U o pcv* II IV* 

jobs and other data 


standardized forms for all 
areas 

profile of results and 
r c Lunine Fi uc t ions > comput- 
erized report 


standardized forms for all 
phases 

3 separate forms used to 
report different results 
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OA/ES 


JEVS 


Prep 


Valpar #17 


omits many details 


specified in detail 


specified in detail 


specified in detail 


oral and reading 


oral and demonstration 


audiovisual 


oral, oral and demonstra- 
Li on ur oral ana ucniun- 
stration with a sample; 
some audiovisual 


not relevant 


minimal 


minimal 


minimal 


not specified 


assistance lowers 
score 


little assistance after 
timing begins 


not specified 


discretion of evaluator 


not recommended 


client request 


strongly recommended 


evaluator times client 


client uses time clock 


evaluator times client or 
cl ient times self 


only one time scor^ used 
on entire system 


varies with type of 
test 


from end of instructions 
to completion of task 


duration of work sample 


preset for the one task 


not used 


rated on 3 point scale 


no separate time norms 


used only for one task 


incorrect answers 
marked 


random check, compared 
to standards 


compared to standards 


except for one area number 

rr\ v r rocnnnCoc 
UT LUircLL re 3 JJUil 3 


computer scored 


minimal use 


some use 


not used 


no separate quality 
norms 


most rated on a 3 point 
scale 


acceptability ratings 


based on total points 


number of correct 
responses 


time and quality given 
equal weight 


qual ity 


number of correct responses 


not used 


16 specific; 4 general 
factors specified 


no factors recorded 


no factors listed 


9 used 


clearly defined 


some factors defined 


some specific areas defined 


5 point rating scale 


3 point rating scale 


3 point rating system 


3 point rating scale 


not specified 


extensive observati?n 


not specified 


not specified 


standardized forms for 
all phases 


andardized forms for 
□hases 


standardized forms for 
all phases 


standardized forms for 
recording and scoring 


computer printout and work 
sample summary of results 


l» Ui Jardized format 
recommends Work Groups 


narrative summary; 
standardized format 


not used; depends upon 
facil i ty 
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Outline 


Career Hester 


McCarron-Dial 


Micro-TOWER 


9. 


utmty 










a. Vocational 
Exploration 


little use to client 


little iiCo 1t\ rl iflnt 


some direct client use 




b. Vocational 

Recommendations 


completely related to 
DOT 


1 of 5 program areas are 
recommended 


related to DOT 




c. Counselor 
Utilization 


designed for counselor's 
use 


disability determination 


designed for counselor use 


10. 


Training in the 
System 










a. Training Required 


yes 


yei 


no 




b. Training 
Available 


yes 


yes 


yes 




c. Duration 


1 day 


3 days 


2 or 3 days 




d. Follow-up 


available 


no*', required 


not specified 


11. 


Technical 
Considerations 










a • liLr III DdSe 


little information 
available 


several groups of 
disabled clients 


19 different norm groups 




b. Reliability 


test-retest relia- 
bilities high 


high .80's; low .90'* 


adequate data In manuals; 
high reliabilities 




c. Validity 

. i 


manual contains very 
little data 


considerable data in 
manuals; separate studies 
in literature 


construct concurrent 
validity reported 
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OA/ES 


JEVS 


Prep 


Valpar #17 


some use to clients 


limited use 


extensive occupational 
information given to 
cl ient 


some direct cl ient use 


occupational group; 
from DOT 


highly related to 
the DOT 


specific jobs and groups 
of jobs 


largely dependent upon 
user 


designed for counselor 
use 


orientated toward 
counselor 


designed for client self- 
interpre tation 


results of each specific 
task designed for coun- 
selor usage 


yes 


yes 


no 


no 


yes 


yes 


yes 


yes 


1 week 


1 week 


3-5 days 


1 day or more 


available 


1 technical assistance 
visit 


yes 


as requested by user 


231 working adults 


1100 clients 


student norms on work 
samples 


"research norms" 


test-re test reliabilities 
given 


no data available 


no data available 


no data available 


no data presented 


no recent data are 
avai 1 able 


content validation only 


no data available 
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Outline 


SAGE 


TAP 


TOWER 


1. Development 

a. Sponsor 

b. Target Group 

r RaCiC nf Ciicfom 

l»« Ddala UT oyStclTl 


Train-Ease Corporation 

students; disadvantaged 
borderline mentally 
retarded 

nflT 
UU 1 


Talent Assessment Programs 

mental levels above train- 
able mentally retarded 

DOT and GOE 


Vocational Rehabilitation 
Administration 

physically and emotionally 
disabled 

job analysis of possible 
jobs for disabled 


2. Organization 

a. Number of Work 
Samples 

b. Grouping of 
Work Samples 

c. Manual 


14 tests, inventories 
and work samples 

4 areas 

separate manual for each 
section; all system 
details given 


10 

each is independent 

single manual ; contains 
all system details 


93 

14 training areas 

single manual ; Some 
details not provided 


3. Physical Aspects 

a. Packaging of 
Work Samples 

b. Durability 

c. Expendable 
Supplies 

u* Rep 1 a Uclilcil L 


all hardware packaged 
individually 

appears durable 

no consumables except 
forms 

from distributor 


all individually packaged 

extremely durable 

no consumable materials 

locally or from distrib- 
utor 


Because ICD does not 

sell hardware, each 
< facility must construct 
'their own. This section 

Is not relevant for 

TOWER 


4. Work Evaluation 
Process 

a. Preliminary 
Screening 

b. Sequence of 
Administration 

c. Client 
Invol vement 

d. Evaluation 
Setting 

e. Time to Complete 
Entire System 


not required 

discretion of eval- 
uator 

1 ittle during testing 
formal testing 
4 hours 


not specified 

8 of the work samples can 
be given in any order 

discretion of evaluator 
not specified 
2*5 hours 


emphasized for planning 

progressive within 14 
areas 

not specified 

realistic work setting 
stressed 

< ynoLf 

0 WccKb 
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Valpar 


Singer 


l 

VIEWS 


VITAS 


Valpar International 


Singer Educational 
Division 


Philadelphia JEVS 


Manpower Administration 


general population, 
Industrially injured 
worker 


special needs population 


mental ly retarded 


educationally and 
culturally disadvantaged 


trait and factor 


groups of tasks in 
related jobs 


DOT 


DOT; 60E 


10 


OA 


1 6 


cl 


each 1s independent 


each is independent 


4 Worker Skill Groups 


16 Work Groups 


separate manual for 
each work sample; most 
material detailed 


single evaluators manual; 
very detailed 


very de ta i 1 ed 


oe ta i lea 


all individually 
packaged 


each self-contained in a 
carrel 


most individually in por- 
table plastic cabinets 


each packaged 
separately 


very durable 


expect some problems 


very durable 


very durable 


few consumable 
supplies used 


wood, wire, chemicals 


paper, string, fiber- 
board 


paper, string, sheet 
metal 


order from developer 


supplies locally or 
through Singer 


all replacement parts 
from developer 


suppl ies local ly; 
parts from developer 


not required 


not required 


not required 


not specified 


discretion of 
evaluator 


discretion of evaluator 


progressive from least 
to most complex 


progressive from least 
to most complex 


minimal 


considerable client 
involvement 


extensive client 
involvement 


considerable cl ient 
involvement 


classroom or work 
place 


classroom atmosphere 


real >stic work setting 
stressed 


realistic work setting 
stressed 


estimate about 1 hour 
per work sample 


2H hours per work 
sample 


20 to 35 hours 


15 hours 
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Outline 

5. Administration 
a. Procedures 



b. Method of In- 
struction Giving 



c. Separation of 
Learning/Perfor- 
mance 

d. Providing Assis- 
tance to Client 

e. Repeating Work 
Samples 



6. Scoring and Norms 

a. Timing 

b. Timing Interval 

c. Time Norms 

d. Error Scoring 

e. Scoring Aids 

f. Quality Norms 



g. Emphasis in 
Scoring 



SAGE 



most specified in 
detail 

sel^-administered or by 
oral instructions 



clear separation 



not specified 



permitted if inval id 
results suspected 



electric and electronic 
timing devices 

specified period of 
time for each task 

converted to 5 or 6 
point scales 



not relevant - errors 
not recorded 



7. Observation of 
Clients 

a. Work Performance 

b. Work Behaviors 



c. Recording 
System 

d. Frequency of 
Observation 



8. Reporting 
a. Forms 



b. Final Report 
Format 
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extensive use 
not relevant 



only number of correct 
responses are recorded 



TAP 



specified in detail 



oral and demonstration 



some 



encouraged for upgrading 



TOWER 



some specified in detail, 
except layout 

written and demonstra- 
tion 



not specified 



not specified 



encouraged for upgrading 



no behavioral observa- 
tional data are 
col lected at present 



standardized forms for 
all phases 

profile of results on 
each area; no narrative 
report 



evaluator times client 



from end of instructions 
to completion of task 

actual time recorded; 
some have error 
penal ties 

errors on some tests 
incorporated into time 
scores 

not used 

some combined with time 
norms for overall score 

time 



no factors defined 
no factors defined 

no rating method used 

not specified 



evaluator times client 



from end of instructions 
to completion of task 

rated on 5 point scale 



compared to standards 

extensive use 

rated on 5 point scale 



time and quality given 
equal weight 



only one factor defined 

a few listed in final 
report 

5 point rating scale 



not specified; frequent 
observations assumed 



two standardized forms 



profile of results on 
specific jobs given 
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standardized form for 
all phases 

narrative report using 
standardized outline 
and ratings 



18 



1 

Valpar 


Singer 


VIEWS 


VITAS 


specified in detail 


specified in detail 


specified in detail 


specified in detail 


oral and demonstration; 
b orne redoing 


audiovisual 


| oral and modeling, flex- 
ibility to lkp a var-- 
iety of techniques 
stressed 


oral and demonstration 


8 work ^amDlp^ Havp 

formal practice 
periods 


1 i ttle 


almost total; well 
established criteria 


no separation 


not specified 


checkpoints built in 


little assistance 
after timing begins 


minimum assistance 


evaluator's decision 


at request of client 


repeated if considered 
necessary 


not recommended 


evaluator times client 


evaluator times client 


evaluator times client 


evaluator times client 


from end of instruction 
to completion of task 


varies with each work 
sample 


after task is learned 
to completion 


after instructions until 
task completed 


dLLUdl tllUc i cLUiUCU ) 

results in percentiles; 
MTM 


ha CdH nn niimkor ot mi 
UaacU U II llUmUci U 1 MINI 

utes to completion 


rAtofi nn nni n t era 1 p 1 

also MODAPTS 


rA tprl nn ^ nnint' cralo 

■ a ICU VJH «J pu MIL JLQ 1 c 


srorpd SPDfl ra tpl v and 
combined with time 
sco re s 


compared to cri teria 


Compared to standards 


compared to standards 


some use 


some use 


some use 


extensive use 


separate norms; per- 
centiles; MTM 


5 point scale or sub- 
tracted from time score 


rated on 3 point scale 


rated on 3 point scale 


weighed combination of 
time and errors 


time and errors given 
equa 1 weight 


time and errors given 
equal weight 


time and errors given 
equal weight 


no factors defined 


20 factors defined 


10 factors defined 


9 factors defined 


17 factors defined 


none listed 


clearly defined 


several general factors 
defined 


5 point rating scale 


none used for behaviors - 
records actual observation; 


specific behaviors 
reported 


specific behaviors 
reported 


not specified 


net specified 


extensive 


almost constant obser- 
vation stressed 


separate form for 
each work sample 


standardized forms for 
all phases 




standardized forms for 
all phases 


standardized forms used 
for all phases 


none used; inde- 

npnrlont wnrt CamnlPQ 
pciiucii l ifUffs aainpicj 


no format given; includes 
Gcbcn p Li vn ut contents 


standard format con- 
Laininy ueriavior oata 
and recommended Worker 
Trait Groups 


standardized format; 

f f i^rif Pnf Unwind 

s tresses worker 
Trait Groups 


0 









Outline 


SAGE 


TAP 


TOWER 


9. Utility 








a. Vocational 
Exploration 


»'ery limited use 


limited use 


exposure to a variety of 
work areas 


b. Vocational 

Recommendations 


1 ists job titles and 
DOT codes 


related to specific jobs 


1 inn* ted to jobs related 
to work areas 


c. Counselor 
Util ization 


dependent upon user 


orientated toward coun- 
selor 


orien ta ted toward counselor 


10. Training in the 
Sys tern 








a. Training Required 


no 


yes 


yes 


b. Training 
Available 


yes 


yes 


yes 


c. Duration 


1 day 


14 days 


2 weeks 


d. Follow-up 


avai lable 


as needed 


no 


11. Technical 

Considerations 








a. Norm Base 


high school students, 
adults 


7 different norm groups 


cl ients 


b. Reliability 


test-rstest; reasonably 
high 


.85 coefficient of 
stability 


no data available 


c. Validity 


data p' sented in manual 


no data available 


equivocal results 



ERIC 



26 

20 



Valpar 


Singer 


VIEWS 


VITAS 


limited use 


extensive information 
given to client 


little use to client 


little use to client 


depends upon use 
in facility 


dependent upon user 


related to DOT 


related to DOT and 
supportive services 


cannot be specified 


dependent upon user 


orientated toward 
counselor 


aimed at counselor 


no 


no 


yes 


yes 


yes 


yes 


yes 


yes 


as needed 


2 day, 1 or 2 weeks 


1 week 


1 week 


available 


avai lable 


1 technical assistance 
visit 


1 technical assistance 
visit 


6 different norm 
groups; MTM norms 


clients, employed 
workers, mim 


452 mentally retarded 

MUUAr lb 


600 CETA clients 


data available; can- 
not be assessed 


test-retest .61 and .71 


no data available 


no data available 


no data available 


mostly content 


no data available 


no data available 
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Outline 


Brodhead-Garrett 


WREST 


WSD 


1. Development 

a. Sponsor 

b. Target Group 

c. Basis of System 


Brodhead-Garrett 

handicapped and 
disadvantaged 

not specified 


Jastak Associates 

severely disabled - 
mentally and physically 

not specified 


Attainment Co. 

severely "mentally dis- 
abled persons" 

«J U03 1 w C VULQ l 1 UNO 1 

skills 


2. Organization 

a. Number of Work 
Samples 

b. Grouping of 
Work Samples 

c. Manual 


18 work samples - 
Phase I 

3 Phases - Phase I - 
sorting, assembly, 
and salvage 

separate manual for 
each phase. Phase I 
lacks many details 


10 

each work sample is 
independent 

well organized manual; 
contains all details 


20 

function and difficulty 
single manual ; most 

Jjr J l»cill Uc La lid y 1 veil 


3. Physical Aspects 

a. Packaging of 
Work Samples 

b. Durability 

c. Expendable 
Suppl ies 

d. Replacement 


Phase I - packaged in 
large wooden cabinet 

very durable 

Phase I - minimal 
amount 

assumed to be from 
local sources 


system packaged in wood 
cabinet 

durable 

mostly paper products 
from developer 


each work sample 
packaged individually 

durable 

forms and plastic bags 

II Q3 pi U» lUcu ) a 1 d u 

order from developer 


4. Work Evaluation 
Process 

a. Preliminary 
Screening 

b. Sequence of 
Administration 

c. Client 
Involvement 

d. Evaluation 
Setting 

e. Time to Complete 
Entire System 


not required 

discretion of eval- 
uator 

assume fairly high 
degree of involvement 

mostly classroom 

reviewer estimates 
Phase I in 1 week 


not required 

discretion of evaluator 

clients told purpose and 
use of results 

formal testing setting 
\h hours 


not specified 

by difficulty 

not specified 

classroom and work 
activity level 

not relevant - training 
stressed 
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Outline 


Brodhead-Garrett 


WREST 


WSD 


5. 


Administration 










a. 


Procedures 


not specified 


specified in detail; can 
be group administered 


specified in detail 




b. 


Method of In- 
struction Giving 


oral & demonstration 


oral & demonstration 


model ing & oral 




c 


Separation of 
Learning/ Perfor- 
mance 


some 


considerable 


minimal 




d. 


Providing Assis- 
tance to Client 


not specified 


none given afier timing 
starts 


discretion of the 
eval uator 




e. 


Repeating Work 
Samples 


permitted to correct 
excessive errors 


encouraged for upgrading 


repeated for upgrading 


6. 


Scoring and Norms 










a . 


Timing 


evaluator times client 


evaluator times client 


not specified 




b. 


Timing Interval 


from end of practice to 
completion of task 


from end of instructions 
for a specified period 
of time 


not specified 




c. 


Time Norms 


reported on 3 point 
scale 


time to completion; com- 
pared to scaled scores 


MTM norms 




d. 


Error Scoring 


not specified 


compared to standards 


compared to standards 




e. 


Scoring Aids 


not used 


not used 


not used 




f. 


Quality Norms 


no s pea rate qual ity 
norms 


all errors totaled for a 
single quality score 


percentage cf errors 




g. 


Emphasis in 
Scoring 


time and qual ity 
given equal weight 


time 


time and accuracy given 
equal weight 


7. 


Observation of 
Clients 










a. 


Work Performance 


no factors defined 


no factors defined 


no factors mentioned 




b. 


Work Behaviors 


36 defined 


10 defined in general 
terms 


no factors mentioned 




c. 


Recording 
System 


5 point scale 


scale from 1 to IB 


no rating system used 




d. 


Frequency of 
Observation 


not specified 


not specified 


not specified 


B. 


Reporting 










a. 


Forms 


standardized forms for 
recording scores and 
work behaviors 


standardized form for re- 
cording performance 


standardized form for 
recording performance 




b. 


Final Report 
Format 


4 page final report, 
topic headings 


numerous examples given 
in manual 


used mostly for training; 
no report format used 
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Outline 


Brodhead-Garrett 


WREST 


USD 


9. Utility 








a. Vocational 
Exploration 

b. Vocational 
Recommendations 

c. Counselor 
Utilization 


extensive,, especially 
with Phase II 

by job area 
not specified 


limited use 
not specified 
not specified 


mainly prevocational 
training device to 
I teach basic skills in 
( assembly, discrimina- 
tion and packaging 

j 


10. Training in tne 
System 








a Tra i n i no Rpou i rpd 


none 


none 


yeb 


b. Training 
Available 


ves 


none 




c. Duration 


2 days to 1 week 


not applicable 


1 day 


d. Follow-up 


as needed 


not applicable 


yes 


11. Technical 

Con <5 "iripPfltion<i 








a. Norm Base 


no data available 


3 major groups; charac- 
tpri<;tic<! wpII ripfikipri 


MTM 


b. Reliability 


no data available 


test-retest coefficients 
in .80's and .90's 


no data available 


c. Validity 


no data available 


correlations between 
scores and supervisor's 
ratings .86 and .92 


no data available 
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The Career Evaluation System 
(Career Hester) 



Development 

a. Sponsor - Originally developed by Dr. Edward Hester of Goodwill In- 
dustries of Chicago, the Hester is presently being marketed by Career 
Evaluation, Inc. 

b. Target Group - The original test battery, now relabeled the Series 
200, was designed to assess physically and mentally handicapped pop- 
ulations. Recently a 17-test modifi cation of the original system 
was developed for use with normal populations with no physical or 
working condition limitations. 

c. Basis of the System - The Career Hester was originally based almost 
exclusively on the third edition of the Dictionary of Occupational 
Titles ; recent revisions in the scoring system have resulted in a 
change to the fourth edition of the DPT. The system emphasizes the 
Data-People-Things hierarchy (DPT), physical conditions, environ- 
mental conditions, general vocational preparation, and specific 
vocational preparation. It must be stressed that the Hester is not 
a work sample system, but a battery cf psychological tests and rat- 
ings designed to relate client scores to the DOT. 

Organization 

a. Name and Number of Work Samples - For the Series 200, 26 separate 
performance and paper-and-pencil tests (for a total of 39 separate 
scores) are used to measure 18 aptitude factors. These scores eval- 
uate the client's abilities on the Data and Things hierarchies. (The 
system requires the evaluator to determine some People levels based 
on interviews, case histories, and behavior observations.) The 26 
aptitude tests are presented below; the equipment or test used to 
obtain each score is given in parentheses: 

Finger Dexterity (Purdue Pegboard), Wrist-Finger Spaed 
(Tapping Board), Arm-Hend Steadiness (Lafayette Motor 
Steadiness Kit), Manual Dexterity (Minnesota Rate of 
Manipulation), Two-Arm Coordination (Two-Arm Tracing 
Apparatus), Two-Hand Coordination (Etch-A-Sketch with 
Maze Overlay), Hand-Tool Dexterity (Hand-Tool Dexterity 
Test), Multiple Limb Coordination (foot operated stapler), 
Machine Feeding (folding machine), Perceptual Accuracy 
(projector with slides), Perceptual Speed (Tachistoscope) , 
Spatial Perception (Revised Minnesota Paper Form Board 
Test), Depth Perception (Lafayette Depth Perception Appa- 
ratus Box) Aiming (Lafayette tiotor Steadiness Kit), Reaction 
Time (Multi-Stimulus Reaction Timer), Fine Perceptual Motor 
Coordination (Polar Pursuit Tracker), Visual Motor Reversal 
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(Mirror Tracing Apparatus), Abstract Reasoning (Raven Pro- 
gressive Matrices), Verbal Ability (SRA Verbal Test - L 
Scale), Nun.srical Ability (SRA Verbal Test - Q Scale), 
Decision Speed (same equipment as Perceptual Accuracy), Re- 
sponse Orientation (same equipment as Reaction Time), Oral 
Directions (Personnel Tests for Industry - Oral Directions 
Test), Reading (Gates-McGinitie Comprehension Test), Arith- 
metic (Level I of the Wide Range Achievement Test), Hand 
Strength (grip dynamometer) and Lifting Ability (standing 
platform) . 

b. Grouping of Work Samples - Eech test is independent; they are not 
grouped during aaministration nor in data recording. 

c. Manual - The 1981-82 revision contains all system details, including 
administration instructions, data entry cc s, sample computer print- 
outs, interpretation of results, and inventc •. Examples of all 
forms are given. The manual is computer printed; some of the sample 
computer printout and forms are almost impossible to read and are 
replaced at no cost. 

Physical Aspects 

a. Packaging of the Work Samples - Each apparatus and test are packaged 
separately for shipping. Because the master control is used for 
several of the psychophysical tasks, the set-up must be changed for 
some tasks. The standardized psychological tests can be ordered 
directly from their respective publishers. 

b. Durability - While no data on the repair or replacement records have 
been made available to the reviewer, he is of the opinion that most 
of the apparatus would be fairly durable if adequately cared for, 

c. Expendable Supplies - The only apparatus tests using expendable sup- 
plies are the folding machine and the foot operated <=tapler. The 
system uses many test forms, answer sheets, etc. 

If the DATAPOINT 1500 series or similar computer is used for on- 
site scoring, other useful computational operations are available: 
bookkeeping, personnel records, client records and word processing. 

d. Replacement - Because of the use of precision apparatus and the need 
for standardized test answer sheets, all replacement parts must be 
ordered from the developer or the respective test publishers. Almost 
no parts can be obtained locally. 

Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. In the 
three week Vegas Evaluation Program designed by and used at Chicago 
Goodwill, the system was administered at the begi.ning of the program 
to determine basic abilities. 
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b, Sequence of Work Sample Administration - With one exception, the 
tests do not have to be given in any specific ord^r. The manual 
recommends that apparatus tests be given first, 

c, Client Involvement - Because of the formal nature of the testing pro- 
cess and the emphasis upon accurate measurement, there is little 
client involvement during actual testing, 

d, Evaluation Setting - The psychometric basis of the Series 200 cre- 
ates, by necessity, a formal testing atmosphere. The emphasis on 
accurate measurement using psychophysical devices to determine re- 
action time, dexterity, etc., results in a laboratory-like 
environment. 

e, Time to Complete the Entire System - The developer estimates that 
the entire battery can be administered in about five hours. The 
individually administered apparatus tests take about one hour to 
administer. The remaining four hours are devoted to paper-and- 
pencil tests that can be administered to small groups. 

Administration 

a. Procedures - For each test the manual gives the purpose, materials, 
test conditions, administration, scoring and data recording. In 
addition, test manuals are provided for all commercially available 
tests. All procedures are thoroughly defined. 

b. Method of Instruction Giving - All instructions are read aloud to 
clients and many are accompanied by short demonstrations. The man- 
ner of cofomuni cation of the instructions may be varied to accommo- 
date any special client problems (e.g., hearing problems, low 
intelligence) . 

c. Separation of Learning/Performance - The manual emphasizes this 
separation and stresses that, within the limits of the test, the cli- 
ent have a full practice period and understand the test. Within the 
limits of the tests per se, this is a practical solution. 

d. Providing Assistance to the Client - The evalu .cor is .0 make certain 
that the client fully understands the instructions for each test. No 
assistance is given during the actual administration of the tests. 

e. Repeating Work Samples - Tests may be readministered if there is 
reason to believe that a client was erratic or he/she was not func- 
tioning at their normal level. Testing may be repeated within two 
or three days of the initial administration. The higher score is 
usually reported. 

Scoring and Norms 

a. Timing - The evaluator times the client, Tlje timing of the psycho- 
physical tests is carefully controlled by electrical timing devices. 
A stopwatch or other timing device is used for the paper-and-pencil 
and dexterity tests. 




b. Timing Interval - For many tests, timing is the speed with which the 
client responds to a specific stimulus by performing highly unique 
responses. A few tests are timed from either start to completion or 
for a definite period of time. 

c. Time Norms - No separate time norms are presented. The raw test 
scores are sent to the developer for computer scoring. 

d. Error Scoring - A few separate error scores are computed. The psycho- 
physical tests use mostly time to completion or the number of re- 
sponses performed within a definite time limit. 

e. Scoring Aids - A few transparencies are used to quickly identify 
correct answers. 

f. Quality Norms - There are no quality scales except in the People Func- 
tions estimates. 

g. Emphasis on Scoring - The emphasis is on time to completion, number 
of responses performed within a definite time limit, or number of 
correct responses. 

7. Observation of Clients 

a. Work Performance - 

b. Work Behaviors - 

c. Rating System - 

d. Frequency of Observation - 

8. Reporting 

a. Forms - Standard forms arc used to record responses on most of the 
psychophysical tests. All data, together with demographic informa- 
tion, are transferred to a final form prior to computer scoring and 
job selection. The raw scores ma w ' be processed in three major ways: 
(1) Mail the data to Career Evaluation Systems, Inc. (2) Telephone 
in the data, which is processed the same day it is received. Basic 
processing costs for the Series 200 are $20.00 and for the Series 
ICj are $15.00. which includes the return first class mailing. Dis- 
counts of up to $4/case on a monthly basis are available for large 
or periodic users. (3) Use of a computer terminal for dvrect access. 
Large users may purchase a minicomputer (64K, 8 ,! disks) or adapt 
software to their existing hardware and use the Series 200 software 
under license. 

b. Final Report Form - "The computer transforms raw test scores into 
scale scores using a 1 to 6 scale. The highest score attainable 
is 6.0 and the lowest is 1.0. M The scale scores are given graphi- 
cally as well as listed baside each score. The printout contains 
four major sections: (1) demographic ?nd identification supplied 
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Because the system is a group of psycho- 
logical and psychophysical tests, no 
behavioral observations are made, ex- 
cept for estimates on the People 
Functions . 



by the evaluator, (2) the scores for each test, (3) the Data-People- 
Things hierarchies showing the client level of functioning, and (4) 
specific DOT job titles sorted from most to least feasible. For 
each job title listed the following information is presented: DOT 
code number, page, OAP Group Code, physical demands, environmental 
conditions, general educational development, and specific vocational 
training. In addition, the Series 100 output also contains a summary 
and evaluation of OAP groups represented in the output. 

9. Utility 

a. Vocational Exploration - The formal testing atmosphere and the lack 
of introductory explanations relating the tests to jobs offers the 
client almost no chance for vocational exploration. 

b. Vocational Recommendations - The major purpose of the system is to 
make specific vocational recommendations. As stated above, the 
printout lists specific job titles that are considered as being 
realistic. 

c. Counselor Utilization - The system is designed to report jobs that 
are within the client's abilities. This information, if communi- 
cated to the referring counselor effectively, could be very useful 
as a realistic basis for client choice, to broaden client vocational 
expectations, to help screen local opportunities and -:o plan train- 
ing for technical vocations. 

10. Training in the System 

a. Training Required - Training is requied prior to use. 

b. Training Available - Training is available either at Career Evalu- 
ation, Inc. or at the purchasing facility. 

c. Duration - One day of free training is provided with the purchase of 
the system. Retraining or training for new staff are also available 
at a cost of $150.00. 

d. Follow-up - Retraining is available upon request. 

11. Technical Considerations 

a. Norm Base - The old 1976 manual states that "norms were developed 
over the years from test scores of stiff members and clients at the 
Goodwill Rehabilitation Center program." However, the norm group is 
not fully described, no rorus are given, no data on the conversion 
process from raw scores to scaled scores are given. The re'ised 
manual does not contain a technical section. In short, there is a 
total lack of information on this phase of the system. 

b. Reliability - Test-retest reliabilities for individual tests on 45 
clients retested after four weeks range from .72 to .95. These are 
high. The second type of reliability is the reproductible ity of the 
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job list. In a test-retest situation, 78% of the job families listed 
on the first printout were the same as those listed on the second. 

c. Validity - There are very little data. The construct validity of the 
system is based on several factor analyses; however, none of these 
are given in the old manual. A concurrent validation study of 156 
dentists demonstrated that 80% of the dentists "would have been rec- 
ommended to enter dentistry." 

12 - Reviewer's Summary and Comments - The system uses the human factors ap- 
proach that has been used as a test development model for over 40 years. 
This approach has proven successful for many psychological tests. The 
Series 200 attempts to present a picture of the client's aptitudes and 
to match these aptitudes with the structure of the DOT. The logical 
structure has a definite appeal to persons who stress aptitude testing 
as part of the vocational evaluation process. It must be emphasized 
that the lack of detailed information on the development and the validity 
of the system is a major source of concern for the Series 200. It is of 
less concern with the Series 100 applied to normal populations, since 
most of the Series 100 tests have standard norms. While descriptions of 
the development process and the scoring procedures are critical for all 
work, sample systems, they are even more critical in a system that handles 
a large mass of data in ways that are not readily available for the 
user's inspection. The lack of details on the process, coupled with the 
almost total lack of technical data, force the potential user to accept 
the results on faith. The system does not claim to be a complete voca- 
tional evaluation system— the developer realizes the need for occupational 
information, interest determination, accurate behavioral observations, 
and evaluator interaction with the client. The system could be best de- 
scribed as a very logical series of tests designed to relate client 
abilities to the Data-People-Things hierarchies of the DOT. The system 
is probably best used for initial screening at the beginning of the 
vocational evaluation process or for when short, yet comprehensive as- 
sessment is needed. 

13. Address 

Career Evaluation Systems, Inc. 
7788 Milwaukee Avenue 
Niles, Illinois 60648 

14. Cost 

The total cost per installed system is $3,250.00 for the Series 100 and 
$7,550.00 for the Series 200. This includes all testing equipment, paper- 
and-pencil tests, data sheets, forms for 100 clients, training, and data 
processing of 10 clients. The Series 100 can be easily upgraded to a 
Series 200 battery. 

15. References 

Botterbusch, K., Hester Evaluation System. In A. Sax (Ed.), Innovations 
in Vocational Evaluation and Work Adjustment. Vocational Evaluation 
and Work Adjustment Bulletin , 1975, 8(4), 62-65" 
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McCarron-Dial Work Evaluation System 
(McCarron-Dial or MDS) 



. Devel opment 

a. Sponsor - The MDS was originally developed by Lawrence T. McCarron 
and Jack G. Dial. It is presently being marketed by McCarron-Dial 
Systems (Common Market Press). 

b. Target Group - The system can be used with: (1) mentally retarded, 
(2) mentally ill, (3) specific learning disabilities, and (4) neuro- 
psychological ly disabled (i.e., higher brain center structural or 
functional disorders). It can also be adapted for use with blind 
and deaf persons. Outside of rehabilitation settings it has been 
used to assess normal and functionally delayed children. 

c. Basis of the System - The system is based on five factors: verbal- 
spatial-cognitive, sensory, motor, emotional, and integration-coping. 
These five factors were derived from an assessment of three dimen- 
sions: verbal and synthetic-spatial skills; sensorimotor skills; and 
emotional -coping skills. 

Organization 

a. Name and Number of Work Samples - The MDS consists of eight separate 
instruments grouped into five factors: 

1) Verbal-Spatial-Cognitive - Wechsler Adult Intelligence Scale (or 
the Stanford-Binet Intelligence Scale) and the Peabody Picture 
Vocabulary Test (PPVT). In many instances, an achievement test 
such as the Wide Range Achievement Test (WRAT) or the Peabody In- 
dividual Achievement Test (PIAT) is also given. 

2) Sensory - Bender Visual Motor Gestalt Test (BVMGT) and the Haptic 
Visual Discrimination Test (HVDT). For visually disabled clients, 
the Haptic Memory Matching Test (HMMT) is used in place of the 
HVDT and the BVMGT. 

3) Motor - McCarron Assessment of Neuromuscular Development (MAND). 
The following ten tasks assess fine and gross motor abilities: 

a) Fine Motor Skills Assessment: Beads-in-Box; Beads-on-Rod; 
Finger Tapping; Nut and Bolt; and Rod Slide. 

b) Gross Motor Skills Assessment: Hand Strength; Finger-Nose- 
Finger; Jumping; Heel -Toe Walk; and Standing On One Foot. 

There are 39 possible scoring options for these ten measures. For 
visually disabled clients, the Adaptation of the MAND for the Vi- 
sually Impaired is available. 

4) Emotional - Observational Emotional Inventory (OEI). In some in- 
stances, the MMPI and the House-Tree-Person are also used. 

5) Integration-Coping - The Dial Behavior Rating Scale (BRS) and the 
Street urvival Skills Questionnaire (SSSQ) are used. 

b. Grouping of Work Samples - The tests, tasks, and scales are grouped 
according to five factors; all devices are closely interrelated. 
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c. Manuals - The system uses a total of four manuals: (1) MDWES-- Eval- 
uation of the Mentally Disabled: A Systematic Approach ; (2) McCarron 
Assessment of Neuromuscular Development Manual; (3) Sensory Integra- 
tion: The Haptic Visual Process es, and (4) Adaptive Behavior: The 
Street Survival Skills Questionnaire . In addition, separate manuals 
are required for the Psychological tests (e.g., PPVT, Bender). These 
manuals contain all system details. The MDWES manual contains an # 
overview of the system as well as data on combining and interpreting 
the results. However, because this manual is relatively detailed and 
statistically sophisticated, several careful readings will be required 
before the reader will be able to fully understand the system. 

3. Physical Aspects 

a. Packaging of the Work Samples - The MDWES is packaged in five sepa- 
rate kits, each about the size of a large briefcase. An auxilary kit 
contains the Bender, PPVT, the OEI, BRS, reporting forms and the sys- 
tem manual. The MAND, HVDT, HMMT, and SSSO are each contained ir. 
separate kits. 

b. Durability - The only component of the MDWES to which the question 
of durability applies is the MAND. Because the evaluator or psychol- 
ogist sets up the equipment and is present at all times, there should 
be little problem with durability. 

c. Expendable Supplies - The only expendable items are the various test 
answer sheets, behavioral observation forms and report forms. No 
consumable materials are used. 

d. Replacement - All replacement parts must be ordered from _ the devel - 
oper. This is absolutely necessary to maintain standardization of 
the testing materials. 

4. Work Evaluation Process 

a. Preliminary Screening - An interview with the client and the referral 
source is urged to obtain background data on the client. 

b. Sequence of Work Sample Administration - Administration begins with 
factor one and continues through factor five. 

c. Client Involvement - Client involvement is encouraged during the as- 
sessment period. Upon completion, the manual recommends individual 
counseling to provide help for the client to move toward realistic 
work-training goals and expectations. 

d. Evaluation Setting - A formal testing setting is used for factors one 
through three and for the SSSQ in factor five. The other two factors 
require a period of placement in a work setting, most commonly a 
sheltered workshop. When used in clinical or educational settings, 
office or classroom situations are used for making behavioral obser- 
vation. 
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e. Time to Complete the Entire System - The first three factors and the 
SSSQ (factor five) can be completed in a day or less. A minimum of 
one week (two weeks are recommended) for systematic observation in a 
work setting, most commonly a sheltered workshop, is required for the 
emotional and integration-coping factors when used in work evaluation. 

Administration 

a. Procedures - Instructions, materials needed, layout and scoring pro- 
cedures are all specified in detail. Standardized tests are ad- 
ministered according to instructions in their test manuals. 

b. Method of Instruction Giving - All instructions for factor three 
(MAND) and parts of factors two (HVDT) and five (SSSQ) are given or- 
ally through demonstration accompanied by kinesthetic cues or total 
communication systems as needed. Factors one and the Bender (factor 
two) are given according to their manuals. 

c. Separation of Learning/Performance - Most of the factors in the MDWES 
are based on formal testing concepts which do not separate learning 
from performance. The MAND allows for repeating of instructions, but 
has no formal criteria as to when the instructions have been learned. 
Because the McCarron-Dial is not a work sample system per se, this 
aspect is not appropriate for most of the MDWES assessments. 

d. Providing Assistance to Clients - The evaluator is to make certain 
that the client fully understands the instructions of each task; no 
assistance beyond that specified by the test manuals can be given 
during formal testing. 

e. Repeating Work Samples - All factors may be repeated as necessary if 
theevaluator questions the accuracy of the results. However, re- 
administration of many of the assessments depends upon the instructions 
in their individual manuals. 

Scoring and Norms 

a. Timing - The evaluator times the client on many tasks; some parts of 
the McCarron-Dit*l are untimed. 

b. Timing Interval - The tasks that are timed generally involve counting 
the number of responses or accurate observation for a specified num- 
ber of seconds. 

c. Time Norms - Separate time norms are given for four or the MAND tests 
(i.e., Beads-in-Box, Beads-on-Rod, Nut-and-Bolt and Standing on one 
foot). The remainder of the tasks (except the WAIS) involve the com- 
bination of time and quality scores to form a single raw score or a 
performance score. These scores are converted to percentiles and 
plotted on a profile sheet. 

d. Error Scoring - The quality of performance is compared to a well de- 
fined set of standards. 



e. Scoring Aids - No scoring aids are used. 

f. Quality Norms - See lf c. Time Norms" above, 

g. Emphasis in Scoring - The system emphasizes the quality of perfor- 
mance. 

Observation of Clients 

a. Work Performance - Work performance factors are identified in various 
rating scales, in the SSSQ* and in the interpretive guidelines pro- 
vided in the MDWES Manual. 

b. Work Behaviors - Work behaviors, as well as personal -social adjust- 
ment behaviors, are clearly specified and many specific work behaviors 
are listed. Most behaviors are defined in observable behavioral 
terms. 

c. Recording System - The OEI and the BRS use a five-point scale to rate 
behaviors and performance factors; the OEI uses a frequency of occur- 
rence scale, while the BRS is a Likert-type scale. Each instrument 
has its own system of combining the individual ratings to form dif- 
ferent scales. 

d. Frequency of Observation - The OEI requires the recording of the 
presence or absence of 50 different behaviors that may manifest dur- 
ing a standard two hour observation period conducted each of five 
days. If the behavior(s) occurs and is further judged to interfere 
with productivity, personal adjustment or work performance of others, 
it is rated. 

Reporting 

a. Forms - Standardized forms are included for the assessment of many 
of the motor tasks and behavioral observations. Summary forms, an 
Individual Evaluation Profile (IEP) and an Individual Program Plan 
(IPP) are also included. The process for scoring and interpretation 
is clearly detailed in a supplemental manual. 

b. Final Report Format - The standard format for comprehensive reporting 
includes specific scores (raw scores, MDS T-scores and z-scores); 
vocational and residential placement scores; behavioral observations; 
case history information; lists of strengths and deficits; program- 
ming priorities; and programming recommendations. The forms which 
may be used for report development are the Individual Evaluation 
Profile (IEP), Individual Program Plan (IPP), Programming Worksheet, 
and the Computer Assessment Program (CAP). The IEP and IPP include 
profile graphs which visually summarize all scores and compare them 
to the general population mean, special population mean, the individ- 
ual's mean, and the vocational program T-score range. Space is 
available in the IPP to include results from other tests and work 
samples. The final report may consist of the CAP, IPP and a summary 
narrative written by the evaluator. 
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The new Computer Assessment Program (CAP) provides an automated in- 
terpretation of McCarron-Dial System input data (raw scores, birth- 
date, and date of evaluation) for prevocational , vocational and 
educational programming. This program has the following features: 
(a) Prints an individual profile of strengths and deficits in fac- 
tors related to vocational and educational programming; (b) recommends 
general vocational and residential program placement; (c) interprets 
the MDS individual assessment data for verbal-spatial-cognitive, 
sensorimotor, and emotional -coping abilities; (d) estimates the prob- 
ability of community employment and suggests general ranges of current 
and anticipated earnings; (e) identifies deficits in comparison to the 
eneral population mean and the individual's mean MDS T-scores; and 
f) prints a programming work sheet that indicates the priority of each 
MDS factor for program planning and training purposes. 

9. Utility 

a. Vocational Exploration - The formalized assessment procedures required 
for the first three factors offer almost no chance for client voca- 
tional exploration. The observation period either in a sheltered 
workshop or on a job site could provide chances for exploration, but 
this depends on the program of each facility. However, the McCarron- 
Dial Occupational Exploration System (OES) is an extension of the 
Computer Assessment Program (CAP). The OES generates a list of DOT- 
coded occupations based on an analysis of the evaluee's factor scores 
obtained from the MDS and WRAT standard scores. The selected occupa- 
tions are listed in the OES report by Worker Trait Group/Occupational 
Aptitude Profile number, DOT number and job title. 

b. Vocational Recommendations - The system assesses the client's ability 
to function in one of the following vocational program areas: day- 
care, work activities, extended sheltered employment, transitional 
training, semi-skilled, skilled, and technical/professional. The 
system also provides guidelines for assessing the client's potential 
functioning in one of five living programs: institutional, inter- 
mediate care, group home, halfway house, and community independent 
living. Examples of final reports for work, educational, and clin- 
ical uses are provided in the various manuals. Emphasis is on 
educational and vocational programming, development and placement. 

c. Counselor Utilization - The system is designed for disability deter- 
mination which includes a description of assets, functional lim- 
itations, and adaptive capacities. The system is aimed at providing 
counselor information and counselor involvement is recommended. 

10. Training in the System 

a. Training Required - A written commitment to pursue training is a 
purchase prerequisite. 

b. Training Available - Basic training is available in work evaluation 
and neuropsychological uses of the MDS. Workshops are held where 
need indicates as well as in Dallas, Texas at the MDS administrative 
office. Advance training is also available on a periodic basis. 
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c. Duration - Each training session takes three days. 

d. Follow-up - No follow-up or site visits are required. They are, how- 
ever, recommended by the developer for maximum use of the system. 

Technical Considerations 

a. Norm Base - Norms for the WAIS, Stanford-Binet, PPVT, MAND and HVDT 
involve 2,000 or more observations each. Norms for the OEI, BRS, and 
SSSQ have been obtained on more than 500 disabled adults each. The 
original normative sample for the entire system as well as the HMMT 
in work evaluation was 200. Additional samples have extended this 
number considerably. Profile revisions occur periodically to reflect 
additional norms. Adult norms on the deaf, blind, and aged popula- 
tions are now available on the HVDT, HMMT, and MAND. Pertinent 
empirical and statistical characteristics of the various norm groups 
are given in the manuals and in research publications. 

b. Reliability - The results of several reliability studies are pre- 
sented in the various manuals. Experimental methods are for the 
most part clearly described. Most data are presented in terms of 
test-retest reliability coefficients and standard error of measure- 
ments. All reliability estimates, except the PPVT, are in the high 
.80's and .90's. 

c. Validity - A variety of validity data are presented for separate 
parts of the system as well as for the entire system. The data pre- 
sented covers mostly construct and predictive validity. The MDS has 
been subjected to several studies (see references) which have demon- 
strated its usefulness as a diagnostic instrument. 

Reviewer's Summary and Comme nts - The McCarron-Dial was designed for the 
purpose of assessing the mentally disabled person's ability to function. 
It uses a combination of widely accepted individually administered psy- 
chological tests, assessments of fine and gross motor ability, and an 
extended period of observation. Rather than discard those tests which 
have proven useful, or to rely solely on performance and behavior obser- 
vation, the McCarron-Dial attempts to combine them into a single pre- 
diction tool. It is encouraging to note that some detailed, well 
designed studies have been conducted with the MDS. In a vocational eval- 
uation setting, the system may achieve its best use as a preliminary 
assessment device for assessing general levels of functioning prior to a 
systematic exploration of interests and specific skills. 

Address 

McCarron-Dial Systems 
P.O. Box 45628 
Dallas, TX 75245 
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Cost 



Auxiliary Component $195.00 

HVDT $390.00 

MAND $490.00 

SSSQ $137.50 

HMMT $567.00 
MAND Adaptations for the 

Visually Impaired $ 28.00 



Manuals are included in the kits but may elso be purchased separately. 
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Micro-TOWER 



Development 

a. Sponsor - The system was developed by the I CD Rehabilitation and Re- 
search Center. However, support was obtained from the HEW Reha- 
bilitation Services Administration to collect normative data, 

b. Target Group - The system was primarily aimed at a general rehabili- 
tation population, but it can also be used with special education 
students, the disadvantaged, and adult offenders. Although not 
specifically designed for mentally retarded persons, it can also be 
used with educable mentally retarded persons. The Micro-TOWER is 
not intended for use with persons who are either above average in 
intelligence or with persons who are trainable mentally retarded. 

c. Basis of the System - The system is basically a group aptitude test 
that uses work sample methodology to measure seven aptitudes as de- 
fined by the fourth edition of the Dictionary of Occupational Titles 
and the General Aptitude Test Battery (GATB). The statistical basis 
are studies on the factor analysis of several work samples and con- 
current validity studies. 

Organization 

a. Name and Number of Work Samples - The system contains 13 work samples 
which measure eight specific aptitudes, plus General Learning Ability 
or G. The work samples are, however, organized into five major 
groups of what can be thought of as second order factors. The pri- 
mary aptitude(s) and the DOT/GATB abbreviation for each work sample 
are given in the parantheses: 

(1) Motor - Electronic Connector Assembly (F-finger dexterity); 
Bottle Capping and Packing (M-manual dexterity); and Lamp 
Assembly (K-motor coordination). 

(2) Spatial - Blueprint Reading (S-spatial reasoning); and Graphics 
Illustration (S-spatial reasoning; K-motcr coordination). 

(3) Clerical Perception - Filing (Q-clerical pe/ception; K-motor 
coordination); Mail Sorting (Q-clerical perception; M-manual 
dexterity); Zip Coding (Q-clerical perception); and Record 
Checking (Q-clerical perception). 

(4) Numerical - Making Change (N-numerical reasoning); and Payroll 
Computation (N-numerical reasoning). 

(5) Verbal - Want Ads Comprehension (V-verbal comprehension); and 
Message Taking (V-verbal comprehension). 

It must be noted that four work samples (Want Ads Comprehension, Zip 
Coding, Blueprint Reading, and Payroll Computation) have alternate 
forms to prevent cheating during administration and for possible use 
during retesting. 

b. Grouping of Work Samples - The work samples are grouped according to 
the five aptitude areas listed above, 
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c. Manual - The system contains several manuals. A general administra- 
tion and scoring manual, a manual for the group discussion program, 
a separate manual for each work sample, a technical manual, and an 
inventory manual. Each of 13 work sample manuals contains the fol- 
lowing: description, materials, setup, administration, scoring, and 
sample forms. All manuals are well written and detailed. 

3. Physical Aspects 

a. Packaging of the Work Samples - All work samples are individually 
packaged; no parts are used by more than one work sample. 

b. Durability - The hardware is durable and because the system uses lit- 
tle complex equipment, minimal equipment replacement can be expected. 

c. Expendable Supplies - Wire in the Lamp Assembly Work Sample and the 
various paper forms are the only expendable supplies. 

d. Replacement - Forms can be ordered from Micro-TOWER or locally dupli- 
cated. The cassette administration tapes must be ordered from the 
distributor. 

4. Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required prior 
to the administration of Micro-TOWER. The manual states, however 
that a period of general orientation to the system should be qiven 
prior to work sample administration. 

b. Sequence of Work Sample Administration - The manual contains several 
suggested schedules for administration of the work samples and for 
group discussion. These schedules are only suggestions and the work 
samples do not have to be given in any set sequence. However, be- 
cause the Wf.nt Ads Comprehension Work Sanple tests the ability to 
read and understand English, it is usually first. Within each work 
sample a carefully defined sequence is followed and all instructions 
to the clients are recorded on a cassette tape. The first step is 
the presentation of a series of occupational photos illustratinq 
jobs requiring the skills assessed by the work sample. Each work 
sample provides an untimed learning/practice period which includes 
taped instructions, visual illustrations, evaluator demonstrations, 
and an opportunity for clients to practice. During this period, the 
cassette tape automatically stops at preselected places so that the 
evaluator can give additional instructions, etc. The evaluator is 
also free to stop the tape at any time if additional help is needed. 
After this learning/practice period comes the evaluation period 
Here clients work entirely on their own without any help. After 
completion of the task, the clients fill out a self-report form rat- 
ing their interest and perceived ability. 



c. 



Client Involvement - Micro-TOWER emphasizes client involvement, which 
is accomplished in several ways. Prior to administration of the work 
sample, occupational information is provided; during the instruction 
period, the evaluator stops at several points to answer questions and 
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provide additional instructions. The practice period also permits 
feedback. The greatest client involvement is during the group dis- 
cussion program. Here, client values, interests, needs, etc., are 
discussed. Suggested activities (e.g., job values, lifelines, choose 
your supervisor) are provided in a separate manual. Clients also re- 
ceive formal feedback of their performances on the work samples. 

d. Evaluation Setting - The evaluation setting could bast be described 
as a combination of a formal testing situation and a group counseling 
environment. The Micro-TOWER is best administered in a room that is 
separate from the rest of the evaluation unit; a "U" shaped table 
arrangement is suggested. These factors add up to the formal testing 
atmosphere. 

e. Time to Complete the Entire System - Total testing time is about 15 
hours; if group discussions are included, the total evaluation takes 
from 19 to 20 hours. Depending on what schedule is used, the battery 
can be administered in between three and five days. The manual con- 
tains several suggested schedules which vary in the number of hours 
per day that the work samples are administered and in the presence 
and duration of the group discussion periods. 

Administration 

a. Procedures - General administration procedures are described in the 
overall manual. The specific manual on each work sample contains 
detailed instructions on materials, layout, administration, scoring 
criteria, etc. All procedures are given in great detail. 

b. Method of Instruction Giving - Instructions are given by several 
methods. Each work sample begins with a series of large photographs 
showing jobs requiring skills related to the work sample. The major 
instructional method, however, is a separate audiocassette tape for 
each work sample which is coordinated with the evaluator's demonstra- 
tions. This tape is programmed to stop at certain critical points 

so that the evaluator can provide help, give additional explanations, 
or check the results of the practice exercises. The system empha- 
sizes standardized instructions and timing and uses the audiotape^ 
as the major means of insuring this standardization. No written in- 
structional materials are used. However, to complete some of the 
verbal and clerical tasks, a third to fourth grade reading level is 
required. In summary, there are five steps in each work sample: 
(1) occupational orientation, (2) basic instructions, (3) practice 
period, (4) timed evaluation, and (5) completion of self-evaluation. 

c. Separation of Learning/Performance - The system places a great deal 
of emphasis on separation of learning from performance. Each work 
sample contains a practice period during which the clients must reach 
certain informal criteria. The evaluation period is timed and is 
only begun once the clients have understood and practiced the task. 

d. Providing Assistance to the Client - Extensive assistance is provided 
during the learning/practice period. None is given during the actual 
evaluation period. 
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Repeating Work Samples - The manual contains no instructions or quide- 
lines for repeating work samples. The only reference to readminis- 
tration is made in regard to the use of alternate forms for four work 
samples. w 

. Scoring and Norms 

a. Timing - The evaluation period on each work sample is timed using a 
cassette tape. The tape tells the clients co "begin," then runs 
through a number of minutes of blank tape and then tells the client 
to stop. This procedure insures accurate timing. 

b. Timing Interval - Timing is for a specified period within each work 
sample. Clients do not continue until they have completed the task. 

c. Time Norms - No time norms are used in this system. The score for 
each work sample is the number of correct responses; report forms 
also provide space for recording the number attempted. 

d. Error Scoring - A separate form is used for each work sample to score 
the number of correct responses, pieces completed, etc. The entire 
product is scored for each work sample; there are no random checks. 
The raw scores for each work sample are recorded on the "Summary of 
Work Sample Performance" sheet. Quality standards are carefully de- 
fined. 

e. Scoring Aids - Some use is made of scoring aids. 

f. Quality Norms - The raw scores for each work sample are compared to 
the desired norm group. A scale is used to convert the scores into 
one of f-e possible ratings. These ratings are based on percentile 
norms (w,nch are given in the technical manual), one rating for each 
20 percentile points. Thus, a very high rating means that the client 
scored above the 81st percentile. Norms are available for 19 dif- 
ferent groups. 

g. Emphasis in Scoring - The emphasis is on the quality of work produced 
within a specified time period. 

Observation of Clients 

a. Work Performance - No specific work performance factors are defined 
in the manual or listed or, the "Behavioral Observations" form. For 
each work sample there is a space for the evaluator to record general 
comments; there are no suggestions in the manual as to what these 
should cover. 

b. Work Behaviors - Five work behaviors (i.e., understanding instruc- 
tions, attention span, work attitude/motivation, need for individual- 
ized help, and efficiency) are listed on the "Behavior Observations" 
form; these are not defined in behavioral terms. This form also in- 
cludes a section on "general behavior" containing items like 
appearance, physical problems and self-image. The evaluator is to 
make short notes for all of the "behaviors" listed on this form. 
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c. Recording System - No rating system is used for any of the items on 
the "Behavioral Observations 1 ' form. However, a six-point scale is 
used for general and work behaviors on the Summary Report Form. 

d. Frequency of Observation - Observations are to be made during the 
training phase, during the performance of the work sample, and during 
group discussion. While no schedule for frequency is specified, it 
would appear that frequent observations are expected. 

Reporti ng 

a. Forms - The Micro-TOWER uses a variety of forms. This includes a raw 
score form for each work sample, the "Behavioral Observations" form 
mentioned above, a "Summary of Work Sample Performance" form, an at- 
tendance form, as well as reporting forms. The client completes a 
"Client Interest and Perceived Performance" form after the completion 
of each work sample; there is also a summary sheet for this form. 

b. Final Report Format - There are three forms used for reporting. The 
first is a profile sheet based on percentiles that gives the client's 
results on each work sample on a scale from "much below average" to 
"much above average." The second is a narrative summary report for- 
mat which may or may not include the forms mentioned under 7. a. 
Finally, there is a "Recommendations" form which uses a checklist 
format to cover topics such as special training, individual attention, 
and vocational recommendations. A computer printout of the percentile 
scores for each work sample is also available. These results are re- 
lated to specific jobs in the fourth edition of the DOT. 

Utility 

a. Vocational Exploration - The information given at the beginning of 
each work sample is designed to make clients aware of what jobs are 
related to the aptitude(s) being measured by the work sample. Many 
of the topics covered in the group discussions center on relating 
personal reeds to job demands and occupational interests. 

b. Vocational Recommendations - The system relates aptitudes to Worker 
Trait Groups that require aptitude patterns similar to those of the 
client. Thus, in making recommendations, the evaluator would match 
client's aptitudes with those required by the Worker Trait Groups. 
This process would be further broken down according to interests, 
interpretations from behavior observations, and the results of group 
discussions. These recommendations would be written in narrative 
form in the narrative summary report. 

c. Counselor Utilization - the Micro-TOWER has two major uses. The 
first is to present a relatively accurate assessment of job related 
aptitudes in a brief period of time. The second is to be a first 
or screening step in an extended period of evaluation. ICD, fcr 
example, uses the Micro-TOWER as a preliminary to the more time- 
consuming TOWER system (Reinert & Loeding, 1978). 
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Training In the System 



a. Training Required - Although formal training is not required, it is 
desirable. 

b. Training Available - Optional training programs are available. 

c. Duration - Two or three days depending upon the training option. 

d. Follow-up - Follow-up is available. 
Technical Considerations 

a. Norm Base - Norms are available on a total of 19 groups. Some of the 
major groups are: general agency rehabilitation clients, males, fe- 
males, Spanish-speaking, left-handed persons, physically disabled, 
psychiatrical ly disturbed, brain damaged, cerebral palsied, students 
in special education, the disadvantaged, recovering drug abusers, 
recovering alcoholics, and adult offenders. Group sizes range from 

40 to over 1300. Most sample characteristics are adequately described. 
Purchasers of the system can receive help from ICD in developing local 
norms. No employed worker norms or industrial standards are used. 

b. Reliability - The Technical Manual provides data on the reliability of 
the Micro-TOWER work samples. The coefficients range from .74 to .97. 
The data was based on test-retest, alternate forms, and internal con- 
sistency estimates. These estimates are very adequate. 

c. Validity - Although a factor analysis revealed a large general factor, 
there was also evidence for grouping the work samples into the five 
aptitude areas. The construct validity of the work sample battery is 
supported by examination of the intercorrelations of the Micro-TOWER 
work samples. Correlations are also available with the factors from 
the General Aptitude Test Battery (GATB). All data are reported in 
the Technical Manual. One study providing positive evidence of 
Micro-TOWER' s use in decision-making compared the recommendations 
made after a one week evaluation with Micro-TOWER to the recommenda- 
tions made after four additional weeks in TOWER. There was a 74% 
agreement on vocational recommendations, suggesting that decisions 

can be reached in a much shorter time for many individuals (Reinert 
& Loeding, 1978). 

Reviewer's Summary and Comments - Micro-TOWER may best be described as a 
group aptitude battery that uses work sampling techniques as the assess- 
ment method. The system claims to measure seven of the nine aptitudes 
that are used in the DOT. The system has the advantage of being group 
administered in a fairly short period of time, thus making maximum use 
of evaluator time. The system attempts to go beyond the mere assessment 
of aptitudes by providing occupational information and group discussion. 
Adequate norms are available, except for employed workers. The system 
generally takes a standardized, psychological test approach with emphasis 
on carefully controlled administration conditions, the separation of 



46 5 2 



learning from performance, and the reporting of results in terms of per- 
centiles. One major problem with the system is the lack of thorough 
behavioral observational materials. Another possible problem is the 
converse of the advantages of a group administered test - the evaluator 
may not be able to provide the client with the one-to-one relationship 
that is needed for some severely disabled persons. 

13. Address 
Micro-TOWER 

ICD Rehabilitation and Research Center 
340 East 24th Street 
New York, New York 10010 

14. Cost 

The cost of the Micro-TOWER depends primarily upon the number of clients 
being tested in the group. Each client requires a complete set of equip- 
ment. An additional set of equipment is needed for the evaluator. Prices 
are available for group sizes from 4 to 30, for example: 

Number of Persons Tested Per Group Pri ce 



The above prices include all equipment, forms to test 100 clients per 
work sample, one set of evaluator' s equipment for each work sample, a 
cassette playback and a cue-stop system, table easels and photo books. 
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Occupational Assessment/Evaluation System 
(OA/ES) 



Development 

a. Sponsor - The OA/ES was designed by Individualized Rehabilitation 
Programs, a private sector rehabilitation service. 

b. Target Group - The OA/ES was initially designed for assessing the 
vocational potential and interests of adults who were disabled as a 
result of accidents or illness. Thus, it was originally intended to 
be used with literate, recently disabled adults who have a work his- 
tory. The developers state that the OA/ES can be used with anyone 
who has the potential for competitive employment. The OA/ES is 
mainly a worker trait group assessment which uses work samples as 
adjunct. 

c. Basis of the System - The entire system is based upon the fourth 
edition of the DOT, the Handbook for Analyzing Jobs, and The Guide 
for Occupational Exploration . 

Organization 

a. Name and Number of Work Samples - The OA/ES consists of paper-and- 
pencil tests, inventories and checklists, apparatus tests, and work 
samples, which are optional. The instruments are as follows: 

Aptitudes : 

Intelligence - The Revised Beta II 

Verbal - Differential Aptitude Test (DAT) Verbal Reasoning 
(Form T) 

Numerical - DAT Numerical Ability (Form T) 
Spatial - DAT Space Relations Test (Form T) 
Form Perception - WAIS Block Design and Picture Completion 
Sub-tests 

Clerical Perception - DAT Clerical Speed and Accuracy 
Motor Coordination - An original apparatus test consisting 

of moving different sized washers from one bolt to another. 
Finger Dexterity - An apparatus test in which 15 small items 
are moved with tweezers from one bin to another. This is 
electronically scored. 
Manual Dexterity - Nuts and bolts are removed from one column 

and placed on another. 
Eye-Hand-Foot Coordination - Operating a small flex shaft and 

foot control in a precision drilling task 
Color Discrimination - Pseudo-Isochromatic es 
Interests and Temperaments - These are assessed uy having the 
client check activities and preferences that are taken di- 
rectly from the DOT. 
Physical Capacity and Environmental Conditions - These are as- 
sessed by the evaluator and are taken directly from the DOT. 
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Work Samples: (1) Systems Planning, (2) File Management, (3) 
Order Request Processing, (4) Product Identification, (5) Grind- 
ing Operations, (6) Drill Press Operations, (7) Electro/Mechanical, 
(8) Welding, and (9) Itinerary Planning 

b. Grouping of Work Samples - All aptitude tests must be administered as 
a unit. Each work sample is independent. 

c. Manual - There are three manuals for the OA/ES. Many system details 
such as specific client instructions and set-up instructions for the 
work samples are lacking. In general, the manuals contain sketchy 
information and omit many system details. The developer, however, 
states that system details are covered in training. A high percent- 
age of the first manual is directly from the Handbook for Analyzing 
Jobs and the DOT. 

Physical Aspects 

a. Packaging of Work Samples - The tests and paper-and-pencil instruments 
are available either in a console or packaged in a large briefcase. 
The nine work samples are packaged as a group. 

b. Durability - The OA/ES unit is constructed out of 3/4 inch pressboard, 
formica laminated. The systems use common tools and provides a one 
year free -from -defects warranty. 

c. Expendable Supplies - Expendable materials include test booklets and 
answer sheets, behavior recording forms for the work samples, and 
some supplies for the drill and soldering work samples. The manual 
contains no estimate of the cost of administration per client. 

d. Replacement - Test items mustT>e ordered from their respective pub- 
lishers. Although not specifically stated, most parts and re- 
placement items should be locally available. 

Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is necessary. How- 
ever, this reviewer estimates that between a sixth and eighth grade 
reading level is necessary to use the complete battery effectively. 

b. Sequence of Work Sample Administration - The work samples follow the 
other measures. Work samples may be given in any order. 

c. Client Involvement - This is not specified in the manual. Judging 
from the purpose of the OA/ES, the administration time, and the 
formal nature of the tests, the reviewer assumes there is little 
client involvement. 

d. Evaluation Setting - The paper-and-pencil tests and the work samples 
lead to a formal testing atmosphere* 
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e. Time to Complete the Entire System - The entire battery, without work 
samples, can be given in "approximately four hours." If all work 
samples were administered, it would require a 25 to 60 hour period. 
A shorter version, the OA/ES II, *hich tests only for aptitudes, oc- 
cupational aspirations, physical ability, and formal education can be 
given in three hours. 

Administration 

a. Procedures - The procedures for the aptitude tests are described in 
general terms. Aptitude tests and work samples do not contain spe- 
cific set-up instructions and administration instructions are very 
vague at times. 

b. Method of Instruction Giving - The paper-and-pencil tests require 
the client to read the instructions as the evaluator reads them 
aloud. Work samples and apparatus tests rely on evaluator oral in- 
structions and demonstration. 

c. Separation of Learning/ Performance - Because the work samples are 
used for vocational exploration, this question is not relevant. 
There is minimum separation in the paper-and-pencil and apparatus 
tests. 

d. Providing Assistance to Clients - The manuals do not contain instruc- 
tions for giving assistance after timing has begun. 

e. Repeating Work Samples - The work samples may be repeated at the dis- 
cretion of the evaluator. 

Scoring and Norms 

a. Timing - During the paper-and-pencil aptitude tests, the client has 
a set period of time for each test. On the apparatus tests, time to 
completion is recorded; many have cutoff times. Note that the work 
samples are used mainly to obtain behavioral data and to provide 
vocational exploration. 

b. Timing Intervals - On the paper-and-pencil tests, there are specific 
time limits. The client is not timed on the work samples but a cut- 
off time is used on some. Timing starts upon completion of the 
instructions. 

c. Time Norms - Time norms are not used with this system. 

d. Error Scoring - On the aptitude tests, all errors are checked against 
a scoring key or with optional computer scoring. 

e. Scoring Aids - Scoring keys are used for the paper-and-pencil tests. 
When computer scoring is used, the following materials are provided: 
(1) summary sheet, (2) DAT score sheets, (3) Revised Beta Examination, 
(4) completed interest, etc., inventories, and (5) color test record 
form. 
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f. Quality Norms - There are no separate quality norms when appropriate 
errors are combined with time scores. See above. 

g. Emphasis in Scoring - The system emphasizes the number of correct 
responses. 

Observation of Clients 

a. Work Performance - No work performance factors are used. 

b. Work Behaviors - The Behavioral Summary Sheet lists nine work "be- 
haviors," such as maturity, convincingness, and frustration, which 
are to be rated on a five point scale. There is an Evaluee Report- 
ing Form which asks five questions about how the client liked the 
work sample; one of these forms is completed for each work sample. 

c. Recording System - For each trait a check is made by the evaluator 
on a five point scale. 

d. Frequency of Observation - This is not specified. 
Reporting 

a. Forms - A Summary Sheet is used for all assessment data, such as apti- 
tudes, interests, temperaments and occupational aspirations. As 
stated above, a Behavioral Surmnary Sheet and Evaluee Reporting Form 
are used during the work sample process. 

b. Final Report Format - There are two final reports. First, the OA/ES 
computer printout gives results of the aptitude tests, interests, 
physical requirements, environment, temperaments, GEO, the client's 
Data-People-Things functions and qualifying Occupational Group 
Arrangements. Second, the work sample Summary of Results presents 
combined responses to the Behavioral Summary Sheet and Evaluee Re- 
porting Form. 

Utility 

a. Vocational Exploration - The nine work samples are intended for the 
client's vocational explanation of the nine DOT Occupation Categories 
(e.g., Machine Trades Occupations, Service Occupations, and Profes- 
sional Technical and Managerial Occupations). Yet the introductory 
material to each work sample does not give any vocational information 
about categories, or groups of jobs. 

b. Vocational Recommendations - The computer printout gives the three 
digit occupation group, followed by its DOT number, and the number 
of specific jobs within the OGA. No specific job titles and DOT 
codes are listed. 

c. Counselor Utilization - Both the manual and the work sample print- 
outs state that this is a guidance instrument and is not to be used 
for vocational placement. Therefore, counselor utilization would 
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depend primarily upon the purpose of the assessment and the occupa- 
tional knowledge of the individual counselor. The printout contains 
a statement that the results are not intended to "restrict career 
opportunities, " 

10, Training in the System 

a. Training Required - Training is required before this system will be 
sold. 

b. Training Available - Training is available from the manufacturer. 

c. Duration - Training time is one week with periodic revaluation. 

d. Follow-up - Follow-up is available after purchase and training. 

11. Technical Considerations 

a. Norm Base - Norms for the aptitude tests were developed on a sample 
of 231 working adults. All sample characteristics are clearly de- 
fined. These are general working population norms. Because the work 
samples emphasize vocational exploration, there are no time and qual- 
ity norms. Finally, the manuals contain no information on the norming 
of the interest and temperaments. 

b. Reliability - The manual contains test and retest reliability coeffi- 
cients for all of the aptitude tests. With sample sizes ranging 
between 106 and 109 and with times between one and six months, the 
test-retest reliabilities range from .86 to .99. Test-retest relia- 
bility coefficients were computed for 20 adults on the interest, 
temperaments, reasons for job satisfaction, and occupational aspira- 
tions. These coefficients range between .69 and .92. Both of these 
sets of data are suspect for several reasons: First, the test-retest 
reliability coefficients for the aptitude tests are unusually high. 
The user should be very suspicious of these. Second, the sample size 
of 20 is much too small to be used in any way other than a pilot 
study. Third, there is no reliability data reported for the work 
samples. In conclusion, these reliability coefficients should not be 
taken at face value. 

c. Validity - The manual states that the tests, work samples, and inven- 
tories are all based on content validation from the DOT. While tests 
taken from the WAIS and the Differential Aptitude Test have been suc- 
cessfully validated in numerous studies, the work sample developer 
presents no information on the content validation for neither the 
aptitude tests nor the work samples. In other words, there are no 
task analyses, job analyses, or any other concrete relationship to 
the world of work. 

12. Reviewer's Summary and Comments - The major advantage of the 0A/ES is 
that the system takes roughly four hours to administer and it is highly 
related to the Dictionary of Occupational Titles . Assessment systems 
having a short administration time and a solid basis in the DOT can be 
readily used by persons in private vocational rehabilitation as well as 
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the more traditional work evaluation settings, however, there are seri- 
ous problems with this evaluation system. First, the manuals lack much 
needed information on the details of administration, set-up, scoring, 
and other procedures. Secondly, the interest inventory is a compilation 
of the 66 Occupational Divisions of the DOT. According to a manual sup- 
plement, when administering the interest instruments the evaluator is to 
read each of the 66 occupational divisions to the client and give exam- 
ples of jobs in each area. This appears to be a cumbersome method of 
administration and is dependent upon the evaluator's examples and expla- 
nations, thus making standardization difficult. In other words, many 
uneducated persons simply would not be able to comprehend this instrument. 
The nine work samples are assumed to represent the nine occupational cat- 
egories in the DOT. It is unreasonable to expect, for example, that one 
drill press operation work sample can represent the thousands and thou- 
sands of occupations in the machine trades category. Finally, the 
problems with the reliability and validity of the system must be weighed 
carefully, especially if the user plans to use the results in testimony. 

13. Address 

Individualized Rehabilitation Programs 

42 West Park Avenue 

Long Beach, New York 11561 

14. Cost 

Standard OA/ES Trait and Factor Battery $3,695.00. 

Deluxe OA/ES Trait and Factor Battery (for group testing of up to five 

people) $4,295.00. 

Combined OA/ES Trait and Factor Battery with work sample package (for 
group testing of up to six people) $5,840.00. 

15. References 
None available. 
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Philadelphia Jewish Employment and Vocational Service 
Work Sample System 

(JEVS) 



Development 

a. Sponsor - Originefly enveloped for the Manpower Administration of the 
U.S. Department <f Labor for use in WIN and CEP programs, the JEVS 
has been refined by the Philadelphia Jewish Employment and Vocational 
Service. 

b. Target Group - Initially designed for the disadvantaged, the system 
has been used in the last several years as an assessment device for 
special needs populations. 

c. Basis of the System - The present basis is the Work Group system of 
the fourth edition of the Di ctionary of Occupational Titles and the 
1979 Guide for Occupational Exploration (GOE) . The philosophical 
basis is a trait-factor approach between common aptitudes and behav- 
ioral demands of the Work Groups and work samples. 

Organization 

a. Name and Number of Work Samples - The JEVS contains 28 different work 
samples. (The purchaser receives a total of 48 separate work samples, 
20 of which are duplicates of the most used work samples.) The 28 
work samples are referenced to 12 Work Groups. Most of the 28 work 
samples are used in more than one Group: 

05.03 - Engineering Technology - Condensing Principle 

05.05 - Craft Technology - Blouse/Vest Making, Pipe Assembly, 

Resistor Reading, Nail and Screw Sort, Lock Assembly, 

Telephone Assembly 

05.09 - Material Control - Computing Postage, Nail and Screw 

Sort, Filing by Numbers 

05.10 - Crafts - Resistor Reading, Telephone Assembly, Metal 

Square Fabrication, Ladder Assembly, Union Assembly 
05.12 - Elemental Work Mechanical - Hardware Assembly, Grommet 
Assembly 

06.02 - Production Work - Telephone Assembly, Hardware Assembly, 

Metal Square Fabrication, Grommet Assembly 

06.03 - Quality Control - Nail and Screw Sort, Collating Leather 

Samples, Nut Packing, Tile Sorting 

06.04 I - Elemental Work: Industrial - Belt Assembly, Grommet 

Assembly, Sign Making, Budgette Assembly 
06.04 II - Elemental Work: Industrial - Collating Leather Sam- 
ples, Nut Packing, Washer Threading, Nut, Bo.t and 
Washer Assembly 

07.02 - Mathematical Detail - Computing Postage, Payroll Compu- 

tation, Adding Machine 

07.03 - Financial Detail - Computing Postage, Payroll Computa- 

tion, Adding Machine 
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07.05 - Records Processing - Filing by Letters, Proofreading, 

Filing by Numbers 
07.07 - Clerical Handling - Filing by Numbers, Rubber Stamping 

b. Grouping of Work Samples - The work samples are organized into 12 
Work Groups for reporting and interpretation purposes. 

c. Manual - The Work Sample Evaluator's Handbook contains detailed admin- 
istration and scoring instructions as well as numerous photographs to 
illustrate proper setup and common errors. 

Physical Aspects 

a. Packaging of the Work Samples - Each work sample is packaged individ- 
ually; no tools or parts are shared with other samples. 

b. Durability - The system uses common tools and materials that should 
be very durable. One possible exception may be the telephone. 

c. Expendable Supplies - In addition to referral, report, and other forms, 
the major expendable supplies are: fabric, paper pads, sheet metal, 
and string. While these supplies should be available locally, they 
can also be purchased from the developer. 

d. Replacement - Most tools and equipment can be locally purchased; other 
items (e.g., colored chips) are available from the developer. 

Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. 

b. Sequence of Work Sample Administration - The work samples ore admin- 
istered in order of complexity beginning with Nuts, Bolts, and Washer 
Assembly and ending with Condensing Principle Drawing. If a client 
is obviousl> not able to complete the work samples at any one level, 
more complex work samples are usually not administered. 

c. Client Involvement - A client orientation is given at the beginning 
of work sampling, a motivational group interview at the end of the 
first day and a structured Feedback interview at the completion. 
Since work sample administration resembles realistic work setting, 
interaction between client and evaluator occurs between work samples 
or during the above sessions. 

d. Evaluation Setting - A realistic work atmosphere and setting are 
stressed in the manual. 

e. Time to Complete the Entire System - The average client takes six or 
seven days for the 28 work samples. 

Administration 

a. Procedures - The layout is clearly described and photographs are used 
to insure proper setup. The materials listed for each work sample 
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are not listed at the beginning of the instructions for that work 
sample. The evaluator is provided with a list of materials for each 
work sample as detailed in the set-up instructions. 

b. Method of Instruction Giving - All instructions are oral and some 
demonstration. Reading is required of the client only when it is a 
requirement in the job area being sampled. 

c. Separation of Learning/Performance - Most of the work samples do not 
have a separate practice period. Typically, the evaluator gives the 
instructions while providing a demonstration. The client attempts 
the task without a prior period of practice. There are no set cri- 
teria to be met prior to timing. Thus, there is minimal separation 
of learning from performance. 

d. Providing Assistance to Clients - Assistance can be given after the 
initial instruction period; but this results in lowering the client's 
final score. The manual contains detailed procedures for providing 
assistance and describes three levels of helping. Each level and 
each type are clearly defined. This emphasis of the analysis of the 
type of assistance is unique to the JEVS system. 

e. Repeating Work Samples - Readministration is not recommended because 
it invalidates results. 

Scoring and Norms 

a. Timing - A time clock is used to stamp the starting and stopping time 
for each work sample. A separate time stamp slip is used for each 
work sample. 

b. Timing Interval - The evaluator punches the time clock after instruc- 
tions are given and the client punches the clock when the work sample 
is completed. 

c. Time Non.:s - Tire results are rated on a three-point scale based on 
the number of minutes to completion. The scale is taken from percen- 
tile scores. 

d. Error Scoring - Most work samples use a random check of items that 
are compared to carefully defined scoring criteria; many use photo- 
graphs to illustrate quality standards. Assistance points are also 
incorporated into the error scoring procedures. 

e. Scoring Aids - Minimal use is made of scoring aids. 

f. Quality Norms - Quality is rated using a three-point scale based on 
the number of counted errors. 

. Emphasis in Scoring - Time and quality are given equal weight. 
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7. 



Observation of Clients 



a- Work Performance - Sixteen specific factors (e.g., size discrimina- 
tion, form perception) and four more general factors (e.g., accuracy, 
neatness) are specified for the system; each work sample has certain 
factors listed that are to be observed. The system stresses the re- 
cording of accurate behavioral observations. 

b. Work Behaviors - The system carefully lists and defines many work re- 
lated behaviors that are to be carefully observed. For example, in 
writiny observations about communication, articulation, tone of voice, 
and grammatical usage are to be noted. Some other behaviors are co- 
operativeness with co-workers and supervisors, reaction to criticism, 
and frustration tolerance. 

c. Recording Systems - Many of the work performance factors are rated on 
a three-point sea 1 *, with all ratings clearly defined and illustrated. 

d. Frequent . ~ r Ssc-rvation - The system uses extensive observations, 
ubserv. efined work factors is required for each work sample; 
these . 'red daily. 

8. Reporti p n 

a. Forms - dardized ortns are included for: reporting the results 
of each wo^k sople. Gaily observation summary, feedback interview 
and a final report. 

b. Final Report Format - The well organized standardized format includes 
some ranking of work sample performance, recommended Work Groups and 
rationale, and extensive written comments on performance and behavior. 

9. Utility 

a. Vocational Exploration - Client vocational exploration is seriously 
limited by two factors: (1) many of the work samples tend to be ab- 
stract, and (2) there is no orientation relating the work samples to 
jobs. 

b. Vocational Recommendations - The final report has a space for two 
Work Groups that are suggested for additional planning. The recom- 
mendations are related to the fourth edition of the DOT and the GOE 
and are geared for both training and job placement. 

c. Counselor Utilization - The system and the final report are oriented 
toward the counselor; however, counselor familiarity with the DOT and 
GOE is necessary for optimal counselor use. 

10. Training Required 

a. Training Required - Yes 

b. Training Available - Yes 
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c. Duration - One week; usually held in Philadelphia. Regional training 
is available under certain conditions. 



d. Follow-up - One technical assistance visit is made to assist with the 
establishment of the system and the maintenance of standardized pro- 
cedures. 

11. Technical Consideration 

a. Norm Base - The system was renormed in 1975 on a total population of 
over 1,100 clients in 32 facilities throughout the U.S. Time and 
quality norms are reported for the total sample as well as separate 
norms by sex, and for different client groups, vocational rehabilita- 
tion, manpower, Goodwill, schools, and special schools for the 
mentally retarded. The norms are given in the 1-2-3 ratings only; 

no means, standard deviations or percentile cutoffs are given. Thus, 
the user has no idea of what the distribution is. Sample character- 
istics are not adequately described. 

b. Reliability - No published data are available. 

c. Validity - Although the initial study of the system gave favorable 
evidence, results of studies done by the U.S. Department of Labor 
have not been released to the public. Research by Nadolsky (1973) 
concludes that the system is valid for evaluation of immediate em- 
ployment potential. There are no recent data available on validation. 

12 - Reviewer's Summary and Comments - The JEVS System is a highly standardized 
and well integrated procedure Tor client evaluation based on 12 of the 
Work Groups of the GOE. The strongest points of the system are its stress 
upon careful observation and accurate recording of work behaviors and per- 
formance factors. The use of a trait-and-f actor approach ties in well 
with the assessment of specific abilities. The major problems with the 
system appear to be the abstract nature of many of the work samples, which 
hinders vocational exploration and the lack of job information presented 
to the client. The system is best used when a thorough evaluation of the 
client's potential is desired. 

12. Address 

Vocational Research Institute 
Jewish Employment and Vocational Service 
1700 Sansom Street, 9th Floor 
Philadelphia, Pennsylvania 19103 

14. Cost 

$8,872.00 includes all work samples, forms, and tuition for training one 
person in Philadelphia. The cost of transportation and living expenses 
for the person to be trained are not included in the price. One on site 
consultation at no charge, other than the consultant's travel expenses, 
may be required. 
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Prep Work Samples 



Development 

a. Sponsor - These work samples were developed by Prep, Inc., and are 
one of the four parts of the Comprehensive Occupational Assessment 
System (COATS). 

b. Target Group - The> were originally designed for use with special 
needs programs. Presently, the system is also being implemented in 
corrections, secondary and vocational education, manpower programs, 
and alternative education programs. 

c. Basis of the System - The 15 Career Clusters identified by the United 
States Office of Education (USOE) are the basis for development of 
the Prep Work Samples. Each of the clusters was subdivided into 105 
job families. The content of the job families is identified through 
job analysis and data contained within the third edition of the 
Dictionary of Occupational Titles . 

Organization 

a. Name and Number of Work Samples - The 27 work samples now available 
are: (1) Drafting, (2) Clerical/Office, (3) Metal Construction, 
(4) Sales, (5) Wood Construction, (6) Food Preparation, (7) Medical 
Services, (8) Travel Services, (9) Barbering/Cosmetology , (10) Small 
Engines, (11) Masonry, (12) Electrical, (13) Police Science, (14) 
Electronics, (15) Automotive, (16) Commercial Art, (17) Nutrition, 
(18) Bookkeeping, (19) Fire Science, (20) Extraction Technology, 
(21) Clothing & Textiles, (22) Real Estate, (23) Communication Ser- 
vices, (24) Refrigeration, (25) Computer Technology, (26) Solar 
Technology, and (27) Machine Trades. 

b. Grouping of Work Samples - Each work sample is independently adminis- 
tered and scored. A summary of results on all work samples is usually 
prepared after the evaluation is completed. 

c. Manual - The general manual describes the theoretical underpinnings 
and information which applies to all work samples. A specific manual 
is provided with the individual work samples. Detailed instructions 
for evaluation points, set-up procedures, participant prerequisites, 
and other relevant information is contained in each specific manual. 
All portions of the manual are contained in a loose-leaf binder. 

Physical Aspects 

a. Packaging of Work Samples - Each of the 27 work samples is separately 
packaged in a portable container. Most nonconsumable supplies are 
stored inside the container. When not in use, the work samples can 
be easily stored. The work samples can be set up on a sturdy table 
or in a carrel . 
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b. Durability - The work sample containers are made of 3/4 inch birch 
plywood and formica. Quality tools and equipment are used (e.g., 
Black & Decker drills; Stanley tools). The Labelle audiovisual 
projectors, when regularly maintained, require minimal repairs. 

c. Expendable Supplies - Each work sample uses consumable supplies. 
These supplies include such items as printed circuit boards, wigs, 
wood, baking items, oil, brake fluid, etc. Consumable packages for 
assessing ten people are available through Prep, Inc. The cost of 
these packages range from $9.00 (Travel Services) to $200.00 
(Commercial Art). The average cost is $65.00 per work sample (or 
$6.50 per client) if purchased through the manufacturer. 

d. Replacement - With the exception of the Individual Report Form, which 
must be ordered from Prep, Inc., all expendable supplies may be pur- 
chased locally. 

Work Evaluation Process 

PreHriiinary Screening - No preliminary screening is required. How- 
ever, the evaluator should check the prerequisites mentioned for each 
mrk sample to avoid unnecessary frustration on the part of the cli- 
ent. 

b. Sequence of Work Sample Administration - There is no predetermined 
sequence of administration. The order and number of work samples 
performed is decided by the evaluator. 

c. Client Involvement - During the assessment process, the evaluator is 
required to make and record behavioral observations as well as per- 
formance observations. These observations are recorded on the 
Individual Report Form. At the conclusion of the work sample, the 
evaluator transfers the client's self-rating of interest, difficulty 
and performance onto the Individual Report Form. Tr.e results of the 
client's and evaluator* s ratings should be reviewed and the client 
given a copy of the Individual Report Form. 

d. Evaluation Setting - The majority of the equipment replicates indus- 
try. However, since the assessment usually takes place in a room 
with tables, carrels, and audiovisual projectors, a classroom atmo- 
sphere prevails. 

e. Time to Complete the Entire System - The average length of time to 
complete one work sample is two hours. The shortest length of time 
needed is 38 minutes for Fire Science, and the longest time is three 
hours and 33 minutes for Electrical. To complete all 27 work samples 
would take an estimated 54 hours. However, in actual use, this many 
wurk samples would not be administered. 

Administration 

a. Procedures - The materials required and the layout are clearly de- 
scribed. 
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b. Method of Instruction Giving - Instructions are presented with an 
audiovisual Labelle format. This format uses an eight track audio 
tape synchronized with a 16mm filmstrip. Automatic stop pulses are 
programed throughout each work sample allowing the client to com- 
plete a performance and work at their own pace. 

c. Separation of Learning/Performance - For each task on which the client 
will be assessed, a demonstration of the task is always given prior 

to the evaluation point. Many times the client is afforded the op- 
portunity to practice the task first. Most client instruction does 
not require a criteria to be reached before proceeding from the 
practice period to the performance period. Thus, there is minimal 
separation of learning/ performance. 

d. Providing Assistance to the Client - If a client is h^Kj difficul- 
ties with a tas* the manual recomnends that the evaluate record 
this difficulty. If the client cannot complete th? v ? vA, the work 
should be evaluated as less than acceptable. The evaludtor may 
assist the client with a task if the remainder of the work sample 

is based upon the successful completion of a task. 

e. Repeating Work Samples - Generally, work samples are not readminis- 
tered unless the client expresses a desire. 

Scoring and Norms 

a. Timing - Time is usually recorded, but it has no bearing on the eval- 
uation except as an indication of severely limited ability. 

b. Timing Intervals - The client is timed during the entire duration of 
the */ork sample. There is available through the manufacturer average 
times for completion for each work sample and possible stopping points 
during each work sample's administration. 

c. Time Norms - The times are reported on the Individual Report Form. 
The evaluator records the total work time. Separate time norms are 
not used. 

d- Error Scoring - After completion, the work sample results are checked 
against carefully defined scorv<g criteria. The results of each 
performance are judged separately on a five pcint acceptability scale. 
An overall rating (high, medium, low) is assigned according to the 
overall acceptability of all performances included in each work sam- 
ple. A rating is also assigned for behavioral observations. 

e. Scoring Aids - Some scoring aids are used. Overlays are used for the 
drawings completed in the Drafting Work Sample. Pages with correct 
responses are supplied with each work sample which requires written 
work. 

f. Quality Norms - The acceptability ratings are based upon industrial 
standards. The performances must be completed within certain toler- 
ances to receive acceptable ratings. Exact tolerances are described 
for each work sample in the specific manuals. 
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g. Emphasis in Scoring - Quality scores for work performance and work 
behavior and the client's self-rating is emphasized. 

Observation of Clients 

a. Work Performance - No work performance factors are listed on the 
Individual Report Form. 

b. Work Behaviors - There are eight behaviors which are observed and 
recorded as part of the evaluation. They are: relationship to 
authority; relationship to co-workers; tolerance for frustration; 
acceptance of criticism; concern for property; work efficiency; re- 
liability; and appropriateness of appearance. Behavioral defi- 
nitions are not presented in the manual. 

c. Recording System - The Individual Report Form describes high, medium 
and low behaviors demonstrated for each of the work behaviors. A 
rating is assigned for each behavior and an average is taken for an 
overall rating. 

d. Frequency of Observation - No predetermined schedule exists. Fre- 
quency will vary according to the number of clients being evaluated 
as well as the frequency of work performance evaluations. 

Reporting 

a. Form - Two forms are used. The Self-Rating Fore allows the client 
to record his/her own interests, difficulties and i>elf-appraisal of 
work. The Individual Report Form is completed by the evaluatG^. 
Information reported includes Evaluator's Rating of Work, Performance, 
Work Rate, and Work Behavior. 

b. Final Report Format - The Individual Report Form becomes part of the 
final report. The evaluator should transfer results from the Self- 
Rating Form. A narrative summary of the results should be written 
at the top of the Report Form. The c ton copy of the Report Form 
should be given to the client. This gives the client access to 
assessment results along with lists of vols and equipment used, 
occupational information, and skills required. A composite of all 
work sample results is usually presented in a narrative format. 



Utility 

a. Vocational Exploration - The client has a wealth of career informa- 
tion available in the work samples. Occupational information is 
presented on the audiovisual cartridge. All tasks are shown in the 
work environment. 

b. Vocational Recommendations - Recommendations could be made after 
assessment and counseling. Primarily, the reconmendations are based 
on job families or individual jobs (DOT) represented by the work 
samples. 
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c. Counselor Utilization - The report forms generated by completion of 
each work sample can be used by counselors and clients alike. As 
mentioned previously, the reverse side of the form contains guidance 
information, The report lists the tasks performed and the degree of 
proficiency reached. 

Training in the System 

a. Training Required - No; but it is strongly recommended. 

b. Training Available - Training is available and can be held either at 
Prep, Inc. or on site. 

c. Duration - One full day of work sample training is sufficient. 

d. Fol lay-up - Yes; customers are regularly offered refresher training 
or follow-up. 

Technical Considerations 

a. Norm Base - The criteria of acceptability used for norming and indus- 
trial quality standards. Client's work is judged against stated 
preestablished levels. Time norms are not used. 

b. Reliability - Since the work samples have been recently revised, new 
reliability studies will be undertaken to determine test-retest cor- 
relations. No data are given in the manuals. 

c. Validity - Work sample validity is based solely on content validity f 
which consists of job analysis and the DOT. The methods are logical 
and result in a group of tasks that represent occupations within a 
particular job grouping. 

Rev iewer's Summary and Comments - The Prep Work Samples are well designed 
modules that can be used with a wide variety of handicapped groups. The 
use of an audiovisual method of instruction permits the presentation of 
occupational information as well as the close monitoring of client prog- 
ress. The manual clearly outlines the tasks that comprise each work 
sample and how each task is assessed. One of the better features of the 
system is that each sample was designed around tasks common to several 
related occupations instead of attempting to duplicate all tasks common 
to only one job. The construction and tools used in the Prep Work Samples 
give the impression of long use if routine maintenance is performed. 
There are two potential problems in using these work samples with a re- 
habilitation population: (1) the use of the audiovisual format may 
present some problems for persons with hearing, visual and/or learning 
handicap., and (2) although designed in part for special needs students, 
the system does not seem appropriate for some lower functioning mentally 
retarded persons. 
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13. Address 



Prep, inc. 

1007 Whitehead Road Extension 
Trenton, New Jersey 08638 

14. Cost 

Prices of work samples range from $480.00 (Fire Science and Police Science) 
to $3,975.00 (Computer Technology); the average price is $1,202.50. Each 
work sample conies with tools, cartridges, specific manual, storage cube, 
and all materials and supplies for assessing 10 clients. The cost of the 
Self-Rating Form and Individual Report Form is $1.25. 

15. References 

Micali, J. J., Comprehension levels of the COATS and Singer Vocational 
Systems: Implications for use with the retarded. Vocational Evalu- 
ation and Work Adjustment Bulletin , 1977, 10(3), 28-31. 

Pisauro, M. L., Comprehensive Occupational Assessment and Training System. 
In A. Sax (Ed.), Innovations in Vocational Evaluation and Work Adjust- 
ment. Vocational Evaluation and Work Adjustment Bulletin, 1976, 9(3), 
39-45. - 
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Pre-Vocation?l Readiness Battery 



(Valpar 17) 



Development 

a. Sponsor - Valpar #17 was developed by Valpar International. 

b. Target Group - The battery is aimed at assessing th± functional skills 
of mentally retarded persons. 

c. Basis of the System - The manual contains no discussion on the basis 
of the system. 

Organization 

a. Name and Number of Work Samples - The system contains five areas, each 
of which has several separate subtests: 

1. Development Assessment - contains four parts which are "simple, 
functional, non-medical measures of physical and mental abilities 11 : 
(a) Patterning/Color Discrimination Manipulation, (b) Manual Coor- 
dination, (c) Work Range/Dynamic Strength/Walking and (d) Matching/ 
Vocational Knowledge/Measurement. 

2. Workshop Evaluation - A simulated assembly process during which 
three clients use a three step assembly process. A fourth person 
(either client or evaluator) acts as an inspector. 

3. Vocational Interest Screening - A sound/slide interest assessment 
in which the client compares two jobs. There are six area scores: 
social service, sales, machine operation, office work/clerical, 
physical sciences, and outdoor. 

4. Social /Interpersonal Skills - This consists of a two page form 
containing descriptions of commonly found barriers to employment. 
Four major areas are covered: (1) personal skills, (2) sociali- 
zation, (3) aggravating behaviors, and (4) work related skills. 

5. Independent Living Skills - An assessment of: (1) transporta- 
tion, (2) money handling, (3) grooming, and (4) living environ- 
ments. The transportation and money handling areas contain three 
levels. Simulation and gaming techniques are used heavily in 
this area. 

b. Grouping of Work Samples - The subtests are grouped according to the 
five areas given above. 

c. There is an overall manual and a separate manual for each area, plus 
a sixth manual that contains norms. Each manual is well organized 
and contains most setup, administration, and scoring instructions. 
The scoring instruction examples are unusuahy detailed. 

Physical Aspects 

a. Packaging of the Work Samples - Equipment for each of the five areas 
is packaged separately. 
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b. Durability - As with other Valpar work samples, all equipment is very 
durable. Construction is fiberboard laminated with formica. 

c. Expendable Supplies - Aside from the numerous forms and recording 
sheets, the system requires no expendable supplies. 

d. Replacement - Forms can be ordered from the developer or may be re- 
produced locally. 

Work Evaluation Process 

a. Preliminary Screening - Subtest One: Developmental Assessment is used 
as a preliminary screening to determine the evaluee's general physical 
strength, mobility, and instruction following skills. 

b. Sequence of Work Sample Administration - Information presented by the 
developer states that the five sections can be given in d.ny order. 
However, because the method of instruction giving for the rest of 
Valpar #17 is at least partially determined during the Development 
Assessment section, this part should be given first. 

c. Client Involvement - While the degree of client involvement with the 
evaluator varies with the section, in general there is a considerable 
degree of client-evaluator contact. Most of the tasks are admini- 
stered individually. The manuals do not contain any discussion on 
procedures for feedback and for sharing the results with the client 
either during or after completion of the five areas. 

d. Evaluation Setting - The setting is not specified. However, the use 
of the various sections implies that a formal testing situation is 
created. 

e. Time to Complete the Entire System - While the time varies with the 
population tested, the general manual estimates 5% hours for the en- 
tire battery. 

Administration 

a. Procedures - Administration procedures layo;", materials needed, and 
general instructions are clearly given in tte manuals. 

b. Method of Instruction Giving - Instructions are given using a variety 
of methods. During the administration of the Development Assessment 
section, the evaluator is first to determine at which of three possi- 
ble levels the client functions: (1) verbal, (2) verbal plus dem- 
onstration, and (3) verbal plus demonstration with a sample to follow. 
The appropriate level is used throughout the remainder of the Valpar 
#17. The Vocational Interest Screening uses a slide/cassette in- 
struction method. Independent Living Skills uses a combination of 
gaming and comparing pictures with accompanying verbal instructions. 

c. Separation of Learning/Performance - In the Workshop Evaluation part, 
some separation of learning from performance occurs; the client is 
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corrected if he/she makes a mistake during the instruction period. 
However, there are no set criteria. This section of the outline is 
not appropriate for the other four parts. Each of these areas is an 
assessment of knowledge and of the ability to learn, 

d. Providing Assistance to the Client - The eva^uator is to make certain 
that the client can perform the task or activity. In most parts, 
extra assistance would not interfere with the results. However, this 
area is one of the few areas not clearly covered in the manuals, 

e. Repeating Work Samples - Readministration is strongly recommended 
when it would facilitate either program evaluation or documentation 
of changes in client skills over a period of time for the purpose of 
adjustment area? of emphasis in training. 

Scoring and Nonms 

a. Time Scores - Except for one task in the Development Assessment Unit, 
no time scores are recorded; all parts are untimed except the Work- 
shop Evaluation. In this part, the number of units assembled in 12 
minutes is the score. 

b. Timing Interval - In the Workshop Evaluation, the 12 minute interval 
begins after the clients have understood the instructions and have 
practiced. 

c. Time Nornis - No time norms are used, except for the one subtest noted 
above. 

d. Error Scoring - All parts are scored on the number of correct re- 
sponses, except Interpersonal-Social Skills. In he Developmental 
Assessment Unit, most tasks are scored by giving either 4, 2, 1 or 0 
points; many of the physical capacity evaluations record the number 
of pounds lifted or moved. The Vocational Interest Screening uses 
the number of choices in the six work areas. The Social -Interpersonal 
Skills uses a negative score in which the more skills and/or behaviors 
that are lacking, the greater the scores in each of the four areas. 
Finally, the numerous scored activities in the Independent Living 
Skills assign one or more points to each correct response. 

e. Scoring Aids - No scoring aids are used. 

f. Quality Norms - The only "quality norms," as this term is used in work 
sample scoring, are those for the Workshop Evaluation. The other four 
sections use norms based on the number of total points. 

g. Emphasis in Scoring - The scoring emphasis is on the number of "cor- 
rect" or appropriate responses. 

Observation of Clients 

a. Work Performance - None of the five sections identify any specific 
work performance factors. 
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b. Work Behaviors - The Interpersonal -Social Skills area contains a sec- 
tion on work related skills. Some of the specific items are (1) 
safety, (2) pronptness, (3) following directions, and (4) work com- 
pletion. Some of these skills are not defined in behavior terms. The 
Workshop Evaluation contains the following four items: (1) corrected 
work, (2) work backed up, (3) on task, and (4) work cohesively with 
others. Each is rated on a three-point scale. Other areas inc^de 
space on forms for writing in general comments and observations. 

c. Recording System - For the Interpersonal -Social Skills, "behaviors" 
are rated using 0, 2, or 4 as a weight. The Workshop Evaluation uses 
a three-point scale of never, sometimes, and consistently. 

d. Frequency of Observation - This point i> really only applicable to the 
Workshop Evaluation section; the manual does not specify the frequency 
of observation. 

Reporting 

a. Forms - Each of the five areas u^s standardized, well -designed forms 
for recording responses and for scoring. Some of the forms include 
black and white reproductions of the stimulus slides and the trans- 
portation gaming exercise. 

b. Final Report Format - This is unspecified due to the wide variety of 
settings and applications for which the work sample was designed. 
However, the Individual Exit Profile provides a sunmary of all scores 
in a manner which transfers to individual education plans used in an 
educational setting. This same form provides a functional summary 
around which the narrative section of most reporting formats can be 
organized. 

Utility 

a. Vocational Exploration - Two parts of the Valpar #17 offer some di- 
rect vocational exploration: Vocational Interest Screening and Work- 
shop Evaluation. The Interest part allows for some exploration and 
provides some occupational information. The Workshop Evaluation, as 
a simulated assembly task, could give the client some concept of pro- 
duction 1 ine work. 

b. Vocational fi^«ommendations - Each of the sections provides data that 
can be use^ io provide vocational recommendations. The specific 
recommendations would be based upon the final reporting format used. 

c. Counselor Utilization - The battery is designed specifically to fa- 
cilitate counseling and/or training after assessment. The scoring 
format specifies goals and potentials by providing counselor insight 
into relative strengths and weaknesses. It also provides score sheets 
with pictorial representations of the work performed in that task to 
remind both the counselor and the evaluee of the activity and perfor- 
mance in each are?. Each subtest also provides both a possible means 
of remediation or training and a format for reassessment to gauge im- 
provements over time. 
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10. 



Training in the Syste m 



a. Training Reor/ircJ - No tr*V.*'.j is required. 

b. Training Available - iFormal training is available. 

c. Duration - At least one day or more, this depends upon evaluator needs. 

d. Follow-up - As requested by the user. 

11. Technical Considerations 

a. Norm Base - A separate norms manual contains what the developers call 
"research norms" on 10 different groups (e.g., competitive employment, 
sheltered workshop, activity center, and homebcund employment). The 
raw scores for each subtest are converted to a single percentile score. 
(The manual contains no information on how this percential score was 
developed.) Over the percentile score on each table are three normal 
distribution curves, which represent three combinations of the norm- 
ing groups. The user can apparently roughly determine where on these 
curves a person falls. Because there are no means, standard devia- 
tions, sample sizes, or descriptions of the samples available, it is 
impossible for this reviewer to make any comment regarding the norms. 

b. Reliability - No data presently available. 

c. Validity - No data presently available. 

12. Reviewer's Summary and Comments - Valpar #17 is intend to be an assess- 
ment of the variables that must be considered when a'^ss a mentally 
retarded person's interests, vocational skills, and vv'\< maturity. The 
system is designed to be used by a person who is fiat grained in psychology, 
medicine, or occupational therapy. The system is well designed, attrac- 
tive, and novel in many ways. The use of audiovisual and gaming materials 
will make it attractive to clients as well as evaluators. Data collection 
forms are unusually well designed. The major problems are in the tech- 
nical areas. The manuals contain no background as to why certain compo- 
nents were selected, no relationship to previous work done in this field. 
No data are given on reliability and validity; there is not even a state- 
ment on these two factors. The norm data are impossible to interpret 
without additional information. In summary, this is a very attractive 
assessment device, but much more needs to be known about it. 

13. Address 

Valpar International 

3801 E. 34th Street, Suite 105 

Tucson, Arizona 85713 

14. Cost - $3,200 

15. References 

None presently available. 
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System for Assessment and Group Evaluation 

(SAGE) 



1. Development 

a. Sponsor - The SAGE was originally developed by Schabacher and Associ- 
ates and Creative Development Associates, Inc.; rights are presently 
owned by the Train-Ease Corporation. 

b. Target Group - The SAGE is aimed at junior high school, secondary, 
and post-secondary students and is especially useful with the disad- 
vantaged. The system can be used with borderline mentally retarded 
and with many handicapped persons with some commonsense modifications. 

c. Basis of the System - The fourth edition of the DOT, the Guide for 
Occupational Exploration and publications* by the Appal acRia Edu- 
cational Laboratory serve as the basis. 

2. Organization 

a. Name and Number of Work Samples - The system consists of four differ- 
ent components that when used together are intended to give the user 
a complete picture of the client on all the major variables contained 
in the DOT. The four components are defined below: 

(1) Vocational Interest Inventory (VII) - This is an untimed paper- 
and-pencil inventory which measures the client's interests in 12 
interest areas taken from the Guide for Occupational Exploration : 
artistic, scientific, plants and animals, protective, mechanical, 
indir '.rial, business detail, selling, accommodating, humanitarian, 
leading-influencing, and physical performing. For each of the 

152 items, the client can make two possible responses— circle one 
response if he/she likes the activity; circle another response if 
he/she has done the activity. Each item uses the following format: 

Puts parts on a car 

Get people to like your ideas 

Make beds in a hospital 

(2) Vocational Aptitude Battery (VAB) - The VAB uses a combination of 
paper-and-pencil tests and isolated trait work samples (i.e., 
apparatus, tests) to measure 11 aptitudes listed in the DOT: 

Verbal - A paper-and-pencil test using a stimulus 
word with four sets of response words. The person 
chooses the response word that is the same or opposite 
in meaning as the stimulus word. 

Numerical - This paper-and-pencil test contains two sub- 
tests— computational items and word problems. 
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General - A paper-and-pencil test, using multiple choice 
format, contains verbal, arithmetic reasoning, and 
spatial items. 

Form Perception - This multiple choice test use': several 
large colored pages of actual tools and mater, .<ls; the 
client must find the tool or material that matches each 
stimulus photograph. 

Color Discrimination - A multiple choice test in which a 
large photograph of standard color samples has been 
mounted on a board. The client must find the color 
sample that matches the stimulus. 

Clerical Perception - Consisting of four subtests, two 
muHiple choice and two performance, two booklets are 
checked to determine if names or numbers are the same; 
cards are sorted by numerical or alphabetical order. 
An ultraviolet light checks for the correct order. 

Spatial - This performance test requires the client to 
reproduce patterns of vaned sized gears on a console; 
the stimulus is a photograph. 

Manual Dexterity - In the test large conduit fittings are 
assembled on a special console. No tools are used; the 
client must use his/her entire hands and wrists to per- 
form the task. 

Finger Dexterity - Small specially modified compression 
unions are assembled on 3/16 inch stainless steel pins. 
No tools are used. 

Motor Coordination - "A manipulative test that requires 
testees to perceive a light flashing on a large elec- 
tronic control board and within a half second period 
of time push or hit a button under the light with either 
hand." One hundred lights flash per minute. This is 
self-timing and self-scoring. 

Eye-Hand-Foot Coordination - Using a test similar to that 
used for World War II and Korean War pilot selection, 
the client uses a stick and foot pedals to match three 
sets of lights showing on a console. The unit provides 
for a trial and test which is self-timed and self-scor- 
ing. 

(3) Cognitive and Conceptual Abilities Test (C-CAT) - This is a mul- 
tiple choice item paper-and-pencil test that measures the six 
levels of General Educational Development (GED). There are 
separate subtests for (1) reasoning, (2) mathematical, and (3) 
language. Results are converted into the scale score from 1 to 
6. 

(4) Assessment of Work Attitudes (AWA) - The AWA is an untimed scale 
containing 30 items covering 20 common work attitude categories 
like workmanship, deferred gratification and persistence as they 
relate to a specific work related situation. A typical item is 
the following: 
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How many people believe in always being on time for work? 



a. 1 out of 5 

b. 2 out of 5 

c. 3 out of 5 

d. 4 out of 5 



The theory underlying the development and inclusion of this in- 
strument in the system is that abilities in and of themselves are 
not sufficient to insure job satisfaction. The prospective em- 
ployee must also bring to the job an appropriate set of attitudes. 

b. Grouping of Work Samples - The tests, attitude scales, and interest 
inventory are grc-iped into the four areas outlined above. 

c. Manual - The loose-leaf binder contains all system details. There is 
a separate section for each of the four parts. Each section contains 
information on development of the section, instructions for clients 
and administrators, scoring, interpretation information, and research 
results. 

Physical Aspects 

<U Packaging of the Work Samples - Each of the testing apparatus is 
packaged independently. 

b. Durability - While no data on durability are available, the plastic 
cases give the impression of being very durable. 

Expendable Supplies - There are no consumable supplies except for 
forms and answer sheets; these may be produced locally. 

Replacement - An 800 telephone number is supplied for replacement or 
service. To maintain standardization, users should check with SAGE 
before using local parts. 

Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is used. The SAGE 
can be considered as the first step in a vocational evaluation. 

b. Sequence of Work Sample Administration - The four components do not 
have to be given in any order. Within the VAB, the aptitude tests 
can be given in any order or on different days. 

c. Client Involvement - There would appear to be little client involve- 
ment during testing; client involvement ?f;.<v testing centers around 
the explanation of a profile fore,-. 

d. Evaluation Setting - The paper-and-pencil and apparatus tests will 
remind the client that he/she- is in a formalized testing situation. 



c. 



d. 
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e. Time to Complete Entire System - The developer states that several 
persons can complete the entire system in about four hours. The sys- 
tem provides for multiple administration; in the VAB there is enough 
equipment/material to assess 23 people simultaneously. 

Administration 

a. Procedures - There is a separate manual for each test and scale. The 
¥A& tests have two manuals— one for electronic scoring and one for 
manual scoring. These manuals are all organized around e common table 
of contents; most system details are specified. 

b. Method of Instruction Giving - Each instrument can either be self- 
administered (if the client reads at the fourth grade level) or by 
having the instructions read aloud. The VII uses a ti tied filmstrip 
and cassette tape which contains instructions as wel"! as the test 

i terns . 

c. Separation of Learning/Performance - The C -CAT and the VAB tests use 
the following procedure— initial instructors, timed practice exer- 
cises, and the actual test. Thus, there is a clear separation of 
learning from performance. One feature of the SAGE is that it con- 
tains a higher percentage of practice items in proportion to the 
actual test items than do most tests. 

d. Providing Assistance to the Client - The manual is not specific on 
this point; apparently the instructor makes certain that after the 
practice exercises there are no probler r A m does not interrupt 
actual test. 

e. Repeating Work Samples - Readministrarirn is permitted if evaluetor 
believes results are not valid. Accoravig lc Lhe developer, the S* 
may be used for pre and post testing. 

Scoring and Norms 

a. Timing - The C-CAT and VAB are timed using an electric timer, con- 
trolled by the evaluator or the client. A buzzer and light mark the 
completion of each timed period. A multi-choice electronic score 
recorder and timer are also available. The Eye-Hand-Foot and Motor 
Coordination tests contain built in timers. 

b. Timing Interval - For each test there is a specific set time for both 
the practice items and the actual test Hems. All testing activity 
is controlled by specific tine lengths. 

c. Time Norms - For the VAB arid C-CAT the number of correct responses is 
converted to a scale score of one to five or six. These scores cor- 
respond to the six aptitude and five GED levels found in the DOT. 
The AWA is scored using a Likert-type scale of from 1 to 4. Item 
values are totaled and the resulting score compared to three scoring 
ranges— job ready, "?," doubtful. In the VII the number of items 
chosen in each of the 12 interest areas are totaled and compared to 
preselected arbitrary cut-off points. 



d. Error Scori. 3 - This is not relevant; no errors are recorded. 



e. Scoring Aids - Scoring keys are used -ith all paper-and pencil tests 
Several units have built in self-timing and scoring devices. The 
Scor-O-Matics provide for self-scoring and timing of all multiple 
choice tests except the C-CAT. 

f. Quality Norms - Not relevant; no errors are recorded. 

g. Emphasis in Scoring - Not relevant because only one type of score, 
a correct response, is recorded. 



Observation of Clients 

a. Nnrk Performan;e 

b. murk Behaviors 

c. Recording System 

u. Frequency of Observation 
Reporti nq 



Presently no procedures for behavior 
observation are available. According 
to the manufacturer, an observation 
form is being developed and will be 
available by late 1982. 



a. Forms - The SAGE uses a ve. ■ a .ty of forms: answer sheets, scoring keys, 
conversion tables, and an obje.tive assessment profile. All forms are 
internally consistent with eacn other. 

b. Final Report Format - The final report is a. profile sheet giving VAB, 
C-CAT, and AWA scores. T he four highest interest areas are also 
listed. Individual oc: nations are listed under each interest area 
and the job demands are compared with thvi abilities of the client. 
Th3re is no narrative report. It must be .'ej/iercbered that the system 
is designed to be used for preliminary assessment and counseling. 

Uti 1 i ty 

a. Vocational Exploration - Hie SAGE provides minimal vocational experi- 
ences; there arc no hands-en materials and little vocational ex- 
planation given prior to tosting. 

b. Vocational Recommendations - The SAGE Objective Assessment Profile 
contains specific job recommendations by DOT title and code. The 
data base for the SAGE contains aptitudes and GED results for 487 
commonly held occupations. An additional manual provides information 
on matching over 12,099 job titles. The system can also be used with 
other commercial systems (there are cross-reference charts relating 
the aptitudes as measured by the SAGE to JEVS, Micro-TOWER, Project 
Discovery, SAVE, Valpar, Singer, VIEWS, and VITAS), with job-site 
training, further testing, etc. 

c. Counselor Utilization - Counselor Utilization would depend upon two 
variables: (1) How the SAGE is integrated with other vocational 
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evaluation tools, and (2) How well counselor knows and uses the DOT 
and related data. A software package (J.O.B.S.) is available for 
on-site job matching. 

Training in the System 

a. Training Required - No training is required. 

b. Training Available - On-site training is available. 

c. Duration - Training is for one day. 

d. Follow-up - Dealer provides toll free number to answer any questions. 
Technical Considerations 

a. Norm Base - VAB research norms are available on r,;ales, remales, handi- 
capped, "normals." Basic statistics are given for Job Corps trainees, 
cooperative education students, males and fen^les. The AWA research 
population was 145 cooperative education students. The C-CAT was de- 
veloped on two groups c f high school students and one group of adults. 
All samples were taken from the mid-Atlantic or southern states; sam- 
ple characteristics ar* fairly well-defined. The major problem is 
that these research norms were enveloped on small samples, in most 
cases of unHer 100. :-'*:-ve^ sr v ; 11 help user develop local norms at 
no charge. 

b. Reliability - Test-retest coefficients and sta^rd errors of measure- 
ment were used to determine reliab-'lUy for most of the tests. Most 
of the KR-20 reliability coefficients are reasonably high. 

c. Validity - A variety of item analyses data and validity data are pre- 
sented for each of the four parts. Validity d^ta includes correlations 
with ratings and other tests; the majority of these are very acceptably 
levels. 

Reviewer's Summary and Comments 

The SAGE is r.ot a traditional work sample system, but rather is a combi- 
nation paper-and-pencil test an^ isolated trait work sample battery which 
can be administered in abCwit Your hours. The SAGE can be used by itself 
o> as a screening device given prior to a vocational evaluation. The de- 
veloper has gone as far as to provide charts of the relationship between 
SAGE and the more popular commercial work sample systems. Because it is 
basically an ability assessment device, the SAGE should bo supplemented 
with appropriate methods of assessing client behavior. Overall, the SAGE 
gives the initial appearance of a well-planned assessment tool that is 
aimed mostly at secondary school students, especially the disadvantaged. 
The major advantages of the system are its relationship to the DOT, its 
relatively short administration time, and *ts apparent flexibility when 
used in combination with other systems. At this time, the major flaw 
relates to sample size. The manual contains detailed descriptions of the 
procedures and methods used to develop each device. While these statis- 
tical methods are very appropriate, the manual uses the term "develop- 
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mental," which is in accordance with the standards of the American 
Psychological Association. It is hoped that the SAGE developers are 
willing to continue data collection and analysis with larger samples. 

Finally, while the SAGE is not specifically designed for i handi- 
capped population, it can be used with most type- of disabilities by 
making a few commonsense modifications. 



13. Address 



Progressive Evaluation Systems Corp. 
21 -aulding Street 
Pleasantville, New York 10" 
' c 00) 431-2016 



14. Cost 



The cost per component is: 

VAB $4,995.00 

VII 500.00 

C-CAT 500.00 

AWA 500.00 



$6,495.00 



Other costs are: 

On-site training (travel expanses not included) $50u 00 

Additional SCOR-0 'MAT ICS 17s'oO 

Additional Electronic Timers 6i).50 

SAGE Carrying Cases 650*00 

J.O.B.S. (Job Opportunity Based Search) 950*00 

15. Ref erences 

Botterbusch, K. F., SAGE - System of assessment and group evaluation 

Vocational Evaluation and Work Adjustment Bulletin , 1982, 15_(1), 
32 "™ 34 • 
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Talent Assessment Programs 
(TAP) 



Development 

a* Sponsor - The system was developed by Talent Assessment Programs of 
Des Moines, Iowa. It is now marketed by Talent Assessment, Inc., 
Jacksonville, Florida. 

b. Target Group - TAP can be used with a wide range of populations and 
all mental levels above trainable mentally retarded. It has oeen 
used with disadvantaged, handicapped and "regular" students and 
adults. 

c. Basis of the System - The results are organized according to specific 
jobs found in the DOT or specific clusters of jobs listed in the 
Guide to Occupational Exploration . 

Organization 

a. Name and Number of Work Samples - Ten tests are included in the sys- 
tem: 

(1) Structural and Mechanical Visualization; (2) Discrimination by 
Size and Shape; (3) Discrimination by Color; (4) Tactile Discrimi- 
nation; (5) Fine Dexterity without Tools; (6) Gross Dexterity without 
Tools; (7) Fine Dexterity with Tools; (8) Gross Dexterity with Tools; 
(9) Flowpath Visualization; ana (10) Retention of Structural and 
Mechanical Deta'l. 

b. Grouping or Work Su.i.ples - Each test is administered and sco^t 1. 
dependently. 

c. Manual - The manual contains general directions, scoring information, 
norms tables as well as several examples of profiles. Specific in- 
structions are given as to materials, set-up, and administration 
instructions for both client and evaluator. Photographs are used to 
insure proper layout. 

Physical Aspects 

a. Packaging of the Work Samples - Each work sample is packaged inde- 
pendently; most of the work samples are contained in plastic cases. 
Optional carrying cases are available; this increases the portability 
of the system. 

b. Durability - The system uses sturdy plastic cases and many of the 
metal components are made of case hardened steel. The TAP tools and 
equipment are extremely durable. 

c. Expendable Supplies - Aside from recording forms, the TAP uses no 
expendable supplies. 
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d. Replacement - If any replacement parts are needed, they could be 
ordered from the developer or purchased locally. The profile form 
and time sheets can be reproduced locally. 

Work Evaluation Process 

a. Preliminary Screening - There is no mention of preliminary screening 
in the manual . 

b. Sequence of Work Sample Administration - Work Sample No, 1 must be 
given first and work sample No. 10 last; the rest may be given in 
any order. The reason for this is that the last work sample requires 
the client to construct the same structure as does the first work 
sample, exce-r: it is done without a model. Thus, the separation is 
needed as a measure of retention. 

c. Client Involvement - The type and degree of client involvement and 
feedback during administration is left to the discretion of the 
evaluator. 

d. evaluation Setting - Although the evaluation setting is not specified, 
the TAP lends itself to a formal testing atmosphere. 

e. Time to Complete the Entire Battery - The tests can be administered 
in from two to two and we-half hours. 

Administration 

a. Procedures - The materials, tools, layout and exact client instruc- 
tions and demonstrations are specified in detail; photographs are 
used for clarification. The mantr,l cautions against testing persons 
who are un c :r medication, ill, depressed, etc, 

b. Method of Instruction Givi"': - While the basic method of instruction 
giving is oral with demonstrations, the evaluator is to "make certain 
that clients have complete understanding of directions" by using 
other techniques, if necessary: "Having clients demonstrate, having 
clients repeat directions, ^nd permitting clients to practice." No 
reading is required for any test. Any spoken language may be used 

in order to instruct the client; sign language may also be used. 

c. Separation of Learning/Performance - There is a client practice 
period prior to timing. After the instructions are given by the eval- 
uator and understood by the client, timing begins. There is some 
separation of learning from performance. 

d. Providing Assistance to the Client - The evaluator is to make sure 
that the client fully understands the task before timing begins. 

e. Repeating Work Samples - To quote the manual: "Assessment should be 
redone if individuals express the fueling that they can do better. 
This will happen with a few who ultimately realize that they did not 
really try." 
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6, Scoring and Norms 



a. Timing - The evaluator times the client using an electronic timer 
that reads in minutes and tenths of minutes, 

b. Timing Interval - Timing begins when the client fully understands 
the instructions and stops when the task is completed, 

c. Time Norms - The actual completion time to the nearest tenth of a 
minute is recorded. After any "penalty" scores ho*'e been added to 
the completion time, the total raw time score is compared to per- 
centile norms. 

d. Error Scoring - Tests 5, 6, 7, and 8 are completion tests. The eval- 
uator checks the task and, if not complete, the client is told to 
complete the task. The additional time needed is recorded and added 
to the original time. Tests 1, 2, 3, 4, 9, and 10 incorporate errors 
into the time secret a penalty time is assessed based on the number 
of errors. This time is added to the raw score and the new score is 
compared to percentile norms. 

e. Scoring Aids - The design of the tests prevents the use of scoring 
aids, 

f. Quality Norms - There are no separate quality norms. In those tests 
which are scored for errors, the number of errors is mul tip! i by a 
constant number and the resulting "penalty" is added to the raw time 
score, 

g. Emphasis in Scoring - Emphasis is on time scores, 

7, Observation of Clients 

a. Work Performance - A few work performance factors are mentioned but 
none are defined; no information is given for their observation, 

b. Work Behaviors - A few work behaviors are mentioned but none are de- 
fined; no information is given for their observation, 

c. Recording System - No method of rating behaviors is used, 

d. Frequency of Observation - This is not specified. Because TAP is 
"Intended to be used primarily as an objective series of tests, the 
system's developer chose not to emphasise client observation, 

8, R eporti ng 

a. Forms - A raw score form and a profile sheet are used, 

b. Final Report Format - The profile sheet contains the percentile scoros 
for each work sample; a placement and training number (P,A,T, #) is 
assigned to each individual profile sheet. This number is assigned to 
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each individual profile sheet based on performance and specific oc- 
cupations in the DOT, worker trait groups in the GOE and vocational 
program areas are recommended for the client through specific cor- 
relations in the manual. 

U tility 

a. Vocational Exploration - Because the system is really standardized 
percept -al and dexterity tests, they are too abstract to prcide 
much direct vocational information to the client without interpreta- 
tion by the evaluator. However, some general occupational infor- 
mation is contained in the manual and this could be given to clients. 

b. Vocational Recommendations - Using the P.A.T. number for each indi- 
vidual profile, the manual li^j. exact job titles with DOT codes in 
conjunction with worker trait groups in the GOE. The job listing 
is comprehensive. 

c. Counselor Utilization - The profile sheet with its occupational rec- 
ommendations is desig^d for the counselor, teacher, employer or 
client. The user has specific information on individual jobs that 
can be used in a variety of ways. 

Training in the System 

a. Training Required - Yes 

b. Training Available - Yes; at the purchaser's site. 

c. Duration - Ab-yjt a day and a half. 

d. Follow-up ' "ornation and consultation can be provided as needed. 
Technical Considerat i ons 

a. Norm Base - Norms are available for: (1) male senior high school 
students; (2) female senior high school students; (3) male junic- 
high school students; (4) female junior high school students; (5) 
a mixed sex group of mentally retarded adults; (6) unselected em- 
ployed young adults; and (7) male alcoholics. All groups arc of 
adequate size, but some details of group characteristics are not 
given. 

b. Reliability - The developers report a coefficient of stability of 
over .85 in preliminary test-retest studies over a six month period; 
however, not enough of the orocedures are given to fully judge the 
meaning of these results. 

c. Validity - No data available. 
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12. Reviewer's Summary and Comments - As opposed to other work evaluation 
systems which attempt to present a complete picture of the client, the 
TAP can be characterized as a battery of perceptual and dexterity tests 
designed to measure gross and fine finger and manual dexterity; visual 
and tactile discrimination; and retention of details. Thus, it is lim- 
ited to the assessment of these fairly specific factors. The developer 
does not claim that this system will assess all vocationally significant 
capacities and behaviors; in fact the manual states that other assess- 
ment devices should be used in addition to the TAP to obtain a complete 
evaluation of the client. 

13. Address 

Talent Assessment, Inc. 
P.O. Box 5087 

Jacksonville, Florida 322C7 

14. Cost 

The $4,125.00 price includes delivery and on-site staff training. 

15. References 

Morley, R. (Ed.), Vocational assessment systems . Des Moines: State of 
Iowa, Department of Public Instruction, 1973. 

Zikmund, D., & Reinders, L., Talent Assessment Program Test Battery, 
(sic) In A. Sax (Ed ), Innovations in Vocational Evaluation and Work 
Adjustment. Vocational Evaluation and Work Adjustment Bulletin , 1974, 
7(4), 58-61. ~ 



ERLC 



35 91 



The TOWER System 
(TOWER) 



1. Developm ent 

a. Sponsor - TOWER was original Iv developed with funding from HEW, Vo- 
cational Rehabilitation Adrm - 'ration. Subsequent refinements have 
been made by the ICD Rehab' ;n and Research Center. 

b. Target Group - Apparently the system was first developed for physi- 
cally disabled persons; it is now used for all types of disabled 
persons, such as emotionally disabled. 

c. Basis of the System - Job analysis of positions that were considered 
open to handicapped persons in the New York City area. 

2. Or ganization 

a. Name and Number of Work Samples - The system contains 93 worl samples 
arranged into 14 job training areas: 

(1) Clerical - Business Arithmetic; Filing, Typing, One-hand Typing; 
Payroll Computation; Use of Sales Book; Record Keeping; and 
Correct Use of English. 

(2) Drafting - T-Square and Triangle; Compass; Working Drawing; Draw- 
ing to Scale; and Geometric Shapes. 

(3) Drawing - Perspective; Forms, Shapes and Objects; Shading; Tone 
and Texture; Color; and Free Hand Sketching. 

(4) Electronics Assembly - ' > lor Perception and Sorting; Running a 10 
Wire Cable; inspecting ,0 Wire Cable; Lacing a Cable; and Solder- 
ing Wires. 

(5) Jewelry Ma;MjV*ct':ring - Use of Saw; Use of Needle Files; Electric 
Drill Press; Piercing and Filing Metals; Use of Pliers; Use of 
Torch in Soldering: and Making Earring and Broach Pin. 

(6) Leathergoods - Use of Ruler; Use of Knife; Use of Dividers; Use 
of Paste and Brush; Use of Scissors and Bond Folder in Pasting; 
Constructing Picture Frame; and Production Task. 

(7) Machine Shop - Reading and Transcribing Measurements; Blueprint 
Reading; Measuring with a Rule; Drawing to Measurement; Metal Lay- 
out and Use of Basic Tools; Drill Press Operation; Fractions and 
Decimals; Measuring with the Micrometer Caliper; and Mechanical 
Understanding. 

(8) Lettering - Lettering Aptitude; Alphabet and Use of T-Square; Use 
of Pen and Ink; Use of Lettering Brush; and Brush Lettering. 

(9) Mail Clerk - Opening Mail; Date-Stamping Mail; Sorting Mail; De- 
livering Mail; Collecting Mail; Folding and Inserting; Sealing 
Mail; Mail Classification, Use of Scale; and Postage Calculation. 



Note: "TOWER" ts an acronym for "Testing, Orientation and Work Evaluation in 
Rehabil < tation. " 
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(10) Optical Mechanics - i^e of Metric Ruler; Use of Calipers; Lens 
Recognition; Lens Centering and Marking; Use of Lens Protractor; 
and Hand Beveling and Edging. 

(11) Pantograph Engraving - Introduction to the Engravograph; Setting- 
Up, Centering Copy and Determining Specified Ratios; Use of Work- 
holder and Adjustment of Cutler; and Setting-Up and Running Off 

a Simple Job. 

(12) Sewing Machine Operating - Sewing Machine Control; Use of Knee 
Lift and Needle Pivoting; Tacking and Sewing Curved Lines; Upper 
Threading; Winding and Inserting Bobbin; Sewing and Cutting; and 
Top Stitching. 

(13) Welding - Measuring; Making a Working Drawing; Identifying Weld- 
ing Rods; Use of Acetylene Torch; Use of Rods and Electrodes; Use 
of Torch and Rod; Measuring and Cutting Metal; and Soldering. 

(14) Workshop Assembly - Counting; Number and Color Collation; Folding 
and Banding; Weighing and Sorting; Counting and Packing; Washer 
Assembly; Inserting, Lacing and Typing; and Art Paper Banding. 

b. Grouping of Work Samples - The work samples are grouped into 14 major 
areas of training. While each of the 14 areas is independent, the 
work samples within each area are arranged in order of complexity. 

In most instances, simpler tasks must be completed before beginning 
complex ones. 

c. Manual - The printed manual is bound in a lnose-leaf folder. There 
is a separate section for each of the 14 areas. Each contains the 
following major headings: orientation preparation, instructions for 
each work sample scoring criteria, and =my scoring aids. Some de- 
tails on the set-up are not included; also it is not always clear if 
instructions should be read to the c^isnt or read by the client. 



Physical Aspects 

a. Packaging of the Work Samples 

b. Durability - 

c. Expendable Supplies 

d. Replacement 

Work Eva! uation Process 



Because ICD does not sell hard- 
ware or equipment, each facility 
> must construct their own. There- 
fore, this information would de- 
pend upon individual facility. 



a. Preliminary Sheening - This is emphasized for planning purposes, but 
the specific information needed pr^or to administration of the system 
is not specified. 

b. Sequence of Work Sample Administration - Administration is progressive 
within the major areas; the choice of areas depends upon client inter- 
est and/or the evaluation plan. 

c. Client Involvement - No client involvement procedures are specified 
in the manua I . 
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d. Evaluation Setting - A realistic wo 3 biosphere and Hting are 
stressed. 

e. Time to Complete the Entire System - The average client completes the 
entire system in three weeks; however, clients seldom take all work 
samples in the system. 

Administration 

a. Procedures - The purpose and procedures are clearly described- All 
tools and materials are listed that are required- Almost no layout 
details are given. 

b. Method of Instruction Giving - The system uses mainly written instruc- 
tions that are supplemented by evaluator explanation and demonstration 
when needed. 

c. Separation of Learning/Performance - Many of the work sample ins u . 
tions do not contain any separation between a formal practice period 
and an established timing period. This manual is not at all clear on 
this point. 

d. Providing Assistance to the Client - The evaluator is encouraged to 
ensure that the client knows how to perform the task before he begins 
to work; procedures for assisting the client after he has started the 
task are not specified. 

e. Repeating Work Samples - The rpariministration of work samples is en- 
couraged for the purpose of upgrading client performance. 

Scoring and Norms 

a. Timing - The evaluator times the client, but no procedure for timing 
is established. 

b. Timing Interval - Timing begins following instruction and stops upon 
completion of the task. Often, however, this point is difficult to 
locate. 

c. Time Norms - Time results are rated on a five-point scale, based upon 
the number of minutes to completion. 

d. Error Scoring - All items are checked against carefully defined scor- 
ing criteria. 

e. Scoring Aids - Extensive use is made cf transparent overlays and other 
scoring aids. 

f. Quality Norms - All work samples are rated on a five-point scale, 
based upon the number of errors. 

g. Emphasis in Scoring - Time and the quality of the finished product are 
given equal weight. 
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Observation of C lients. 

a. Work Performance - The only work performance factor specifically listed 
is "dexterity." 

b. Work Behaviors - A comprehensive checklist of work bavhaiors (e.g., 
neatness, attendance) are listed in the vocational evaluation report. 

c. Recording System - A five-point system is used to rate "work and per- 
sonal characteristics"; the points on the scale are not clearly 
defined. 

d. Frequency of Observation - Frequent observations are not emphasized, 
but are taken for granted. There is no established procedure for be- 
havior observation. 

Reporti ng 

a. Forms - Standardized forms are used for attendance and punctuality; 
for a summary of time and quality results for each work sample; and 
for a "vocational valuation report." 

b. Mnal Report Format - The three page final report contains ratings of 
-Work and Personal Characteristics," ratings for each of the 14 job 
areas and a narrative report. 

Utility 

a. Vocational Exploration - The client is exposed to many different train- 
ing areas which are representative of a variety of jobs. The manual 
contains some specific occupational information that is given during 
the administration of the work samples. 

b. Vocational Recommendations - Vocational recommendations are limited 
to jobs that are directly related to the work samples. The recorrmen- 
dations are not high^ related to the DOT and are primarily training 
orientated. 

c. Counselor Utilization - Counselor involvement in the evaluation pro- 
cess is recommended; the final report is aimed at the referring 
counselor and client. 

Training in the System 

a. Training Required - Yes, for inexperienced vocational evaluators. 

b. Training Available - Yes; this includes training in other work sample 
systems as well as work sample development. 

c. Duration - Two weeks 

d. Follow-up - No 
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11 • Technical Considerations 



a. 



Norm Base - The system was normed on clients at the Institute for the 
Crippled and Disabled (ICD): sample sizes or characteristics are not 
given. Industrial norms are not available. 

b. Reliability - No data available. 

c. Validity - A seven city research study produced equivocal results 
The empirical validity of the TOWER is still open to much question. 

12 ' ^viewer's Summary and Comments - The TOWER System is the olchst complete 
work evaluation system and over the years has served as a model for the 
development of many work samples. The TOWER uses a realistic job settinq 
to thoroughly evaluate clients for a rather narrow group of jobs The 
facts that the TOWER was based on job analysis and that the system has 
been used for many years to place and crain handicapped people are indi- 
cations that the system is very useful in evaluating clients for a small 
group of jobs. The lack of precise definitions for work performance fac- 
tors and client behaviors and the lack of adequate norms are the major 
weaknesses of the system. The high use of written instructions and the 
high level of the areas evaluated restricts its use with low literate and 
mentally retarded clients. 

13. Address 

International Center for the Disabled 
340 East 24th Street 
New York, New York 10010 

14. Cost 

The TOWER Evaluator's manual is priced at $200.00 and includes copies of 
all work samples, response sheets and scoring criteria. The work samples, 
plus three extra sets of evaluations are available individually at prices 
ranging from $25.00 to $100.00. Training tuition is $450. .0, which in- 
cludes manual and all software. Note: No hardware is sold b* ICD; each 
facility constructs the work samples. ICD estimates cost to set up unit 
at about $5,000.00. 

15. References 

Backer, T., Methods of assessing the disadvantag e d in manpower pr ograms: 

A review an d analysis . Los Angeles: Human Interaction Research 

Institute, 1972. 

Backer, T. , Client assessment: A manual for employment and t raining 
agencies . Los Angeles: Edward Glaser & Associates, 1979. 

Board of Education of Baltimore County, Mobile unit to provide vocation al 
evaluation for handicapped children . Towson, Maryland: Author, 1973. 

Burrows, J., The state and private agency view of the role of the pre- 
vocational- unit in the community. In New York University, School of 
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Valpar Component Work Sample Series 
(Valpar) 



Development 

a. Sponsor - The work samples were developed or modified by Valpar In- 
ternational . 

b. Target Group - The work samples were originally intended for use with 
the general population but have been used extensively with industri- 
ally injured workers. The manuals do not contain a statement that 
the work samples are designed to serve any specific population; it 
can be assumed that the Valpar can be used with a wide variety of cli- 
ent groups. Modifications are available for Valpar 1, 2, 3, 4, 7, 8, 
9, and 10 for use with the visually handicapped. Videotapes and 
signed administration instructions are available for deaf persons on 
Valpar 1 to 16, except 14. 

c. Basis of the System - According to the developers, the work samples 
are based on a trait-and-f actor approach taken from job analysis. The 
manual for each work sample relates that work sample to several Worker 
Trait Groups Arrangement as well as specific occupations. 

Organization 

a. Name and Number of Work Samples - At present there are 16 work samples 
contained in the series: 

(1) Small Tools (Mechanical); (2) Size Discrimination; (3) Numerical 
Sorting; (4) Upper Extremity Range of Motion; (5) Clerical Comprehen- 
sion and Aptitude; (6) Independent Problem Solving; (7) Multi -Level 
Sorting; (8) Simulated Assembly; (9) Whole Body Range of Motion; (10) 
Tri-Level Measurement; (11) Eye-Hand-Foot Coordination; (12) Solder- 
ing and Inspection (Electronics); (13) Money-Handling; (14) Integrated 
Peer Performance; (15) Electrical Circuitry and Print Reading; and 
(16) Drafting. 

b. Groupina of Work Samples - The work samples were developed and are in- 
tended for use as individual components and are not grouped as an 
evaluation system. 

c. Manual - A separate manual is used for each work sample. Each con- 
tains sections on purpose, job classifications, work sample descrip- 
tion, general administration and scoring, client instructions, rating 
directions, and normative data. A separate Evaluator's Manual con- 
tains sections on scoring norms for each work sample as well as 
descriptions of the norm groups and methods. Most material is de- 
tailed and easy to follow. 
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. Physical Aspects 

a. Packaging of the Work Samples - ATi work samples are packaged separ- 
lockab?" cases" 6d ' appropriate » work samples have 

b. Durability - Components are well -constructed and durable, requirinq 
little or no maintenance. One exception may be the Money Chanqinq 
work sample where problems with the dial can occur. 

c. Expendable Supplies - Most of the Valpar work samples require no ex 
pendable supplies. The few that do use mostly paper forms. 

d. Replacement - All replacement parts can be ordered from Valpar. Forms 
may be reproduced locally or ordered from Valpar. 

Work Evaluation Process 

a. Preliminary Screening - The work samples do not require preliminary 
screening. J 

b. Sequence of Work Sample Administration - The order and the number of 
work samples to be given is left to the discretion of the evaluator 

11 ni« be / emembered that the Valpar is a group of independent work 
samples and not a system. 

c. Client Involvement - Because work sample administration resembles a 
formal testing situation, client involvement is minimal; feedback on 
ual evaluator S ^ UP t0 ^ discretion of tne fa cility and individ- 

d. Evaluation Setting - The work samples can be used in either a class- 
room setting or workshop setting. However, noise levels, liqhtinq 
and placement of the work samples are recommended. 

e. Time to Complete Entire System - It is estimated by the reviewer that 
most work samples can be completed in one hour or less. The Draftinq 
Integrated Peer Performance, and Clerical Comprehension and Aptitude 
could take over one hour to administer. 

Administration 

a. Procedures - The materials required, evaluator instructions, and the 
layout are clearly described in the manual; detailed illustrations of 
the work sample are used to insure accuracy. All work samples re- 
cycle themselves so that they ready for the next administration, 
inus, nttle evaluation time is spent in disassembling completed 

tdSKS • 

b. Method of Instruction Giviijg - A combination of oral instructions 
with accompanying demonstrations is used by the evaluator to admin- 
ister most work samples. Instructions are read verbatum from the 
manuals. In the Clerical Comprehension and Aptitude and the Money 
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Handling Work Sample, the client is required to read instructional 
and testing materials that simulate the tasks required in these two 
work samples. 

c. Separation of Learning/Performance - Eight of the Valpar work samples 
have a formal practice period during which time the client must reach 
an established criteria. 

d. Providing Assistance to the Client - The evaluator is encouraged to 
insure that the client has a thorough understanding of the task and 
demonstrate each task, if necessary, on those work samples without 
formal practice sessions before beginning timing. The manuals do 
not specify what (if any) assistance may be given to the client after 
timing has started. 

e. Repeating Work Samples - Readministration is encouraged if desired by 
the evaluator. 

Scoring and Norms 

a. Timing - The evaluator times the client. On some work samples (e.g., 
Clerical Comprehension and Aptitude), where there are several distinct 
tasks, each task is timed separately. The disassembly of many work 
samples is also timed. The manuals are specific as to when timing 
should begin and end. 

b. Timing Interval - Timing begins after the instructions have been given 
and ends when the task is completed. There are no cutoffs in terms 

of time-to-completion., except in Simulated Assembly. 

c. Time Norms - The completion time in seconds is recorded for each por- 
tion of all work samples. The total time is converted into percentiles 
at 5% intervals; MTM standards also use percents as a conversion 
method . 

d. Error Scoring - Errors are well defined; the number of errors is re- 
corded for each part of the sample and totaled. Total errors are 
converted to a percentile score. The Valpar also uses a performance 
percentile score which is a combination of time and error scores. 
Where appropriate, there are MTM error norms for all work samples. 

e. Scoring Aids - Use is made of scoring aids; some work samples have 
automatic scoring devices. 

f. Quality Norms - Separate quality norms are used; errors are converted 
to a percentile form at 5% intervals. 

g. Emphasis in Scoring - The emphasis is on the performance percentile 
score, which is weighed in combination of time and error scores. 

Observation of Clients 

a. Work Performance - No work factors are specified for individual work 
samples. 




b. Work Behaviors - The same 17 worker characteristics (e.g., ability to 
work alone; ability to respond to change; ability to communicate; 
ability to make decisions) are defined in each work sample manual; 
there are no behaviors that are to be observed for each separate work 
sample. Most of these characteristics are not clearly defined and 
all require subjectivity on the part of the evaluator. Evaluators 
are instructed to rate only those characteristics "which are applica- 
ble to the client." 

c. Recording System - The evaluator uses a five-point scale to rate cli- 
ents on each of the 17 worker characteristics. 

d. Frequency of Observation - Frequency of observation is not specified; 
however, frequent evaluator contact is required on many work samples 
due to the administration and scoring procedure. 

Reporting 

a. Forms - A separate standard form is used for each work sample for re- 
cording scoring information and rating worker characteristics. Body 
position charts are included with the Upper Body Range of Motion and 
Whole Body Range of Motion work samples for recording pain and fa- 
tigue. 

b. Final Report Format - Because the work samples are not part of a uni- 
fied system, no information or recommendations are given for reporting 
results in a unified manner. 

Utility 

a. Vocational Exploration - There is limited opportunity for vocational 
exploration due to the abstract nature of some of the work sample. 

b. Vocational Recommendations - Because these are individual components 
and not a system evaluation, vocational recommendations cannot be 
made on the basis of one work sample. The use of the Valpa* work 
samples for making vocational recommendations largely depends upon 
their use by the individual evaluation unit. 

c. Counselor Utilization - Because the system uses the purchasing facil- 
ity's report format, counselor utilization cannot be specified. 

Training in the System 

a. Training Required - Training is not required as a condition of pur- 
chase. 

b. Training Available - Training is available from Valpar International. 

c. Duration - This duration depends upon the needs of the evaluator. 

d. Follow-up - Follow-up after training is available on a consultation 
basis. 
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Technical Considerations 



a. Norms - Each of the work samples were normed on the following groups: 
(1) Two groups of mentally retarded persons; (2) Air Force enlisted 
personnel; (3) employed workers; (4) deaf persons; (5) skill center 
trainees and (6) community colleges. Sample sizes for each group are 
a minimum of 50 and range upward to over 550. All samples are clearly 
described. Means and .standard deviations are given for time, error, 
and performance scores for each group. Results are given in percen- 
tile. Methods-Times-Measurement (MTM) standards (or norms) were 
developed for each work sample. 

b. Reliability - The test-retest reliability for each part of eacfi work 
sample is given. The standard error of measurement was also computed. 
The reliability coefficients are generally very high. Because the 
methods used to gather and analyze the data are not given, no assess- 
ment can be made about the meaning of these data. 

c. Validity - Each manual contains short descriptions of the different 
types of validity. However, no data are available. 

Reviewer's Summary and Comments - The Valpar Component Work Sample Series 
currently consists of 16 individual work samples which are physically 
well designed and constructed. They are appealing to clients and lend 
themselves to easy administration and scoring. Individual work samples 
can be easily incorporated into an existing evaluation program. Because 
these individual work samples can be purchased as needed by facilities, 
there are no unified final report forms and other aspects of an integra- 
ted system are lacking. The major problem with the Valpar is in the area 
of relationship to jobs. According to the manuals, each component is 
keyed to a number of specific occupations as well as worker trait groups. 
However, the manuals offer no convincing evidence that, for example, one 
work sample could be related to ten Worker Trait Groups. 

Address 

Valpar International 
3801 East 34th Street 
Tucson, Arizona 85713 

Cost 

Individual work samples range from $575.00 to $1,525.00. Any number of 
work samples can be purchased. 
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Vocational Evaluation System by Singer 
(Singer) 



Development 

a. Sponsor - The system was developed by the Singer Educational Division, 
Career Systems. 

b. Target Group - According to the manual, "The VES is primarily intended 
for special needs populations (e.g., socially and educationally dis- 
advantaged, mildly retarded, physically handicapped) but may also be 
used with essentially normal populations. Those special needs groups 
who have limited reading ability, test poorly, and have a lack of oc- 
cupational experiences ..." Thus, it appears that the Singer 
developers feel the system could be used with a wide range of rehabil- 
itation, educational and manpower populations. 

c. Basis of the System - The work samples within the system are based on 
a group of tasks contained in closely related jobs. The basis is a 
combination of job analysis procedures and the job descriptions con- 
tained in the fourth edition of the Dictionary of Occupational Titles . 

Organization 

a. Name and Number of Work Samples - Presently the following 24 work 
stations are available: 

(1) Sample Making; (2) Bench Assembly; (3) Drafting; (4) Electrical 
Wiring; (5) Plumbing and Pipe Fitting; (6) Woodworking; (7) Air 
Conditioning and Refrigeration; (8) Sales Processing; (9) Needle 
Trades; (10) Masonry; (11) Sheet Metal Working: (12) Cooking and 
Baking; (13) Small Engine Service; (14) Medical Service; (15) Cos- 
metology; (16) Data Calculation and Recording, (17) Production 
Machine Operating; (18) Household and Industrial Wiring; (19) Filing, 
Shipping and Receiving; (20) Packaging and Materials Handling; (21) 
Electronics Assembly; (22) Welding and Brazing; (23) Office Services; 
(24) Basic Laboratory Analysis. 

b. Grouping of Work Samples - Each work station is independent. 

c. Manual - The present Singer manual contains two basic sections. First, 
the technical section contains data or reading level, a job-task ma- 
trix, job analysis, and norms for each work sample. Second, the 
administrative section provides general information on client behav- 
ior, and how to complete all of the forms used. The administration 
section for each sample contains the following topics: tools and 
materials, set-up and maintenance, administration, scoring, and qual- 
ity standards. The manual is unusually thorough and very well 
organized. A planned revision to the current manual will result in 
fiye separate smaller manuals: technical, administration, installa- 
tion, orientation, and report writing. 
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Physical Aspects 



a. Packaging of the Work Samples - Each work station is self-contained 
in a carrel that is closed and locked when not in use. Some larger 
pieces of equipment such as a box-and-pan break and microwave oven 
are located outside of the carrel. 

b. Durability - Because the Singer stations use fairly sophisticated 
tools and equipment,, it is expected that there would be some problems 
with durability. 

c. Expendable Supplies - Many of the stations use a considerable amount 
of expendable items. For example, the Sheet Metal Working station 
requires two fairly large squares of metal, hinges, a latch and pop 
rivets. Other stations require wood, wire, fabric, and baking in- 
gredients. Based on Singer price lists of January, 1982, the average 
price of expendable supplies is $3.52 per job sample. The range is 
between $0.33 and $9.48. 

d. Replacement - All expendable supplies as well as most of the tools 
can be locally obtained. Other parts can be ordered from Singer. 

Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. 

b. Sequence of Work Sample Administration - The order and the number of 
work stations given is left to the discretion of the evaluator. 

c. Client Involvement - The client is involved in the evaluation process 
through a series of self-ratings on interest and performance. Due 

to the frequent evaluator checkpoints in each work sample, the possi- 
bility for client contact with the evaluator is high. The manual does 
not specify if formal feedback is to be given to the client at the 
end of the evaluation process. 

d. Evaluation Setting - The use of the carrels and audiovisual instruc- 
tions could not help but to create a school -like atmosphere. 

e. Time to Complete the Entire System - The manual states that M a gen- 
eral Rule of Thumb is to allow two to two and one-half hours per job 
sample." Because any number of stations may be administered, no 
realistic estimates of the length of time to complete the total 
system can be given. 

Administration 

a. Procedures - The tools and materials needed as well as set-up and 
maintenance are given in the manual for each work station. All cli- 
ent instructions are given on auto-vance equipment. 

b. Method of Instruction Giving - All instructions are given using an 
audio-cassette tape and filmstrip format with the client control 1 if q 



the rate of advancement. Typically, the client hears several frames 
of instruction, then turns off the equipment, performs a specific 
task and then calls the evaluator to check that task. The linear 
programmed material is occasionally supplemented with written mate- 
rial* Additional evaluator instructions are discouraged because they 
would interfere with the standardization; evaluators are to record 
any type of re-instruction. 

c. Separation of Learning/Performance - There is very little separation 
of learning from performance in the Singer system. Most client in- 
structions do not require a criteria to be reached before going on 
to repeat the task on a timed basis. On four of the stations, the 
client completes a product, has it checked by the evaluator, and then 
performs the task again without instructions on a speeded basis. 

d. Providing Assistance to the Client - The evaluator is encouraged to 
make sure that the client knows how to do the task before he begins 
to work; checkpoints are provided in the audiovisual p^;erial so 
that the client can ask the evaluator to review his progress before 
continuing. 

e. Repeating Work Samples - Work samples may be repeated at the "request 
of the client who expresses a desire to try to improve his or her 
performance." The evaluator may have a work station repeated to as- 
sess changes in performance. 

6. Scoring and Norms 

a. Timing - The client is timed by the evaluator. For each work station, 
the manual contains instructions at the frame number (s) where the 
evaluator is to start and stop timing. 

b. Timing Interval - The interval varies with each work sample and is 
specified for each work sample in the manual. In many work samples, 
there are several timing intervals. 

c. Time Norms - All norms are based on the number of minutes to complete 
the work sample. Participant and/or employed worker norms are re- 
ported using a five-point rating scale based on the time score distri- 
bution for each work sample. Methods-Time-Measurement (MTM) norms 
are reported in 10% intervals, with industrial normal being 100%. 

d. Error Scoring - All errors are carefully defined and each item (or 
the entire finished product) is checked against the criteria. In 
using the MTM standards for quality, each error is classified as ma- 
jor, intermediate, or minor. 

e. Scoring Aids - Some use is made of scoring aids. 

f. Quality Norms - Participant norms are reported using a five-point rat- 
ing scale. Industrial norms and MTM quality norms are based on 100% 
with a specified number of percentage joints subtracted for each of 
three levels of errors. 

g* Emphasis n Scoring - Time and error are given equal weight. 
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Observation of Clients 



a- Work Performance - Twenty work factors (e.g., attention span, form 
discrimination, neatness and use of hand tools) are defined. Each 
work sample has a separate Task Observation Record which contains 
specific factors for each task. For example, in the part of the 
Drafting Work Sample that "compares drawings with models provided, 11 
the evaluator is to observe M fol lowing a model, inspection and check- 
ing, and retention." 

b. Work Behaviors - No work behaviors are listed. 

c. Recording System - Work performance factors are listed on the Task 
Observation Record; the evaluator does not rate behaviors, he records 
the observations. A Work Activity Rating Form is used by the client 
to rate his/her interest in a work station before and after perform- 
ing the tasks; the client and the evaluator also rate the client's 
performance on a five-point scale at the end of the work sample. This 
rating is general and does not include separate ratings for work fac- 
tors. 

d. Frequency of Observation - The manual lists frequent evaluator check- 
points and assistance points which allow the evaluator to make nu- 
merous observations at each station. 

Reporting 

a. Forms - Forms include the Task Observation Record, Work Activity Rat- 
ing Form, MTM Rating Form, Industrial Rating Form and a summary sheet 
for time and quality scores. 

b. Final Report Format - While the manual does not contain any recommended 
final report format, it does contain a description of what should be 
contained in a final report. 

Utility 

a. Vocational Exploration - An extensive amount of occupational informa- 
tion is provided to the client; each work sample contains an intro- 
duction to some jobs related to the work sample. Many schools and 
facilities use the Singer primarily as an interest and career explo- 
ration device. 

b. Vocational Recommendations - Because the system contains no final re- 
port fonnat, it is difficult to judge the type and quality of vocational 
recommendations. These would depend upon the user. 

c. Counselor Utilization - For the reason given above, this aspect cannot 
be accurately judged. 

Training in the System 

a. Training Required - No 
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b. Training Available - Yes 



c. Duration - Two day, one-week or two-week VES workshops are offered on 
a regional level on a fee basis. 

d. Follow-up - Singer regional managers conduct follow-up visits without 
charge. Technical consultations may be scheduled through the home 
office. 

Technical Considerations 

a. Norm Base - Each Singer unit contains three types of norms: client 
(i.e., participant) norms, employed worker norms, and MTM. All norm 
groups are of adequate size and sample characteristics and thoroughly 
described. 

b. Reliability - A study by Cohen and Drugo (1976) reported test-retest 
reliability coefficients of .61 and .71 for an EMR population. These 
correlations are moderately reliable. 

c. Validity - The Singer bases its validity on several sources. First, 
the content validity of >e job-task matrix and of the job analysis 
for each work sample. The job-task matrix relates specific tasks to 
specific jobs and identifies which tasks are included in the work 
sample. The average work station covers about 65% of the tasks given 
in the matrix. Second, two predictive studies (Gannaway and Sink, 
1972; Monroe County, n.d.) attempted to relate work sample scores 
with success in jobs related to the work samples. While these studies 
have methodological problems, the significant results are encouraging. 
Third, a study by Sink, et al. (1976) revealed that the system en- 
couraged users to seek additional occupational information. 

Reviewer's Summary and Comments - In a review of the Singer system pub- 
lished in an earlier version of this publication, the system was criticized 
for its inadequate manual. The most recent Singer manual corrects for the 
earlier lack of thoroughness and goes on to provide almost all the data 
that the evaluator would need. The process of developing MTM and employed 
worker norms is one of the strong points of the VES. Also encouraging is 
the publication of some basic studies on the system. Presently the system 
provides a measure of interest measurement and skill assessment for jobs 
mostly in the skilled trades and technical areas. The occupational infor- 
mation remains the strong point of the system. The major problems are the 
lack of work atmosphere, the use of expendable supplies, and the possible 
need for a superstructure to integrate the units into a functional whole. 

Address 

Singer Educational Division 
Career Systems 
80 Commerce Drive 
Rochester, New York 14623 
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14. Cost 



As of May, 1982, cost per work station ranges from $1,190.00 to $2,590.00, 
with the average cost being $1,650.00. The price includes shipping and 
enough supplies to evaluate approximately 30 people. 
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Vocational Information and Evaluation Work Samples 

(VIEWS) 



Development 

a. Sponsor - Philadelphia Jewish Employment and Vocational Service. 

b. Target Group - The system is especially designed for mild, moderate, 
and severely mentally retarded adults. 

c. Basis of the System - The VIEWS is based on six Worker Skill Groups 
falling within four Data, People, Things levels of the fourth edition 
of the Dictionary of Occupational Titles . These levels were chosen 
because they represent the most common areas of training and employ- 
ment for mentally retarded persons. 

Organization 

a. Name and Number of Work Samples - The 16 work samples are organized 
according to Worker Skill Groups. These groups are Vocational Re- 
search Institute constructs which represent a class of exercises 
involving similar task demands. 

(1) .687 - Materials Sorting - #1 Tile Sorting, #2 Nuts, Bolts & 
Washers Sorting, #8 Valve Disassembly; Clerical Matching & Count- 
ing - #5 Stamping, #10 Mail Sort, #11 Mail Count; Assembling - 

#4 Collating & Stapling, #6 Nut Weighing, #7 Nut, Bolt & Washers 
Assembly, #9 Screen Assembly 

(2) .686 - Machine Feeding - #12 Machine Feeding 

(3) .685 - Routine Tending - #3 Paper Cutting, #16 Drill Press 

(4) .684 - Fabricating - #13 Budgette Assembly, #14 Valve Assembly, 
#15 Circuit Board Assembly 

b. Grouping of Work Samples - The work samples are grouped according to 
the four D.P.T. levels listed above. 

c. Manual - The manual contains the following information for each work 
sample: demonstration, setup, training, production and norms. A 
photograph is used for each work sample to insure proper setup. The 
use of the recording forms, report forms, etc., is not covered in the 
manual. These are dealt with during training. 

Physical Aspects 

a. Packaging of the Work Samples - Fourteen work samples are individually 
packaged in portable plastic cabinets. The Drill Press and Machine 
Feeding Work Samples are permanently mounted on a sturdy worktable. 

b. Durability - The plastic cabinets as well as the components for each 
work sample appear to be very durable. 
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c. Expendable Supplies - Several sizes and colors of paper, string, and 
fiberboard squares are the only expendable supplies used. These all 
can be purchased locally. 

d. Replacement - In order to insure standardization, all replacement 
parts should be ordered from the developer. 

Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. 

b. Sequence of Work Sample Administration - The work samples are given 
from least complex to most complex. Each work sample has three 
phases: (1) Demonstration - the e valuator follows the manual to pro- 
vide an oral description and a physical demonstration for the client; 
(2) Training - the client is trained to a predetermined criterion of 
mastery on each work sample— during this phase the e valuator is free 
to use a wide variety of techniques to make certain that the client 
learns the task; and (3) Production - after the criterion have been 
achieved, the client is assigned a set number of cycles of the work 
sample to perform independently. The purpose in separating the train- 
ing and production phases is to make sure that the client has learned 
each task before he performs it. 

c. Client Involvement - There is extensive client involvement. In the 
training phase for each work sample, the e valuator and the client 
have a significant amount of interaction during the learning process. 
The E valuator's Handbook calls for an informal client feedback ses- 
sion after the first day as well as on subsequent days when needed. 

d. Evalu^tljn Setting - A realistic work atmosphere and setting are 
stressed in the Handbook and during evaluator training. 

e. Time to Complete the Entire System - The developer estimates that the 
VIEWS can be administered in from four to seven, five hour days (i.e., 
20 to 35 hours). 

Administration 

a. Procedures - The Handbook contains all details necessary for adminis- 
tration. A photograph of each work sample is used to insure proper 
layout. The instructions for the demonstration phase are given in 
detail and include both oral and physical directions. The training 
phase criteria are clearly given. 

b. Method of Instruction Giving - No reading is required of the client 
for any work sample. The demonstration phase uses oral instructions 
plus modeling. During the training phase the evaluator is free to 
use a variety of verbal and nonverbal techniques; flexibility is 
stressed here. Because each work sample is individually administered, 
the client can receive instructions using the methods which best meet 
his needs. 




c. Separation of Learning/Performance - As stated above, the VIEWS sepa- 
rates learning and performance by having a formal training period for 
each work sample. Here the evaluator is free to use almost any teach- 
ing technique that will result in the client reaching the established 
criteria. For example, the criterion for the Valve Disassembly Work 
Sample is: "Two valves consecutively disassembled and sorted without 
error." 

d. Providing Assistance to the Client - Extensive assistance is provided 
during the training phase; little is given during the production 
phase. If help is needed during the production phases, the evaluator 
is to record this on the appropriate behavior observation form. 

e. Repeating Work Samples - The VIEWS does not place much emphasis on 
repeating work samples; it is designed so that the client should have 
learned the task before the performance phase. However, work samples 
may be repeated if considered necessary by the evaluator. 

Scoring and Norms 

a. Timing - The evaluator uses a time stamp machine to time the client. 

b. Timing Interval - Timing on each work sample begins when the client 
enters the production phase and ends with the completion of the task. 

c. Time Norms - Time results are rated on a three-point scale based on 
the number of minutes to completion. Predetermined time standards 
using the MODAPTS approach are also available. 

d. Error Scoring - Each work sample is checked against carefully defined 
quality standards. No random check is made; the entire work sample 
is scored. 

e. Scoring Aids - Some use is made of scoring aids. 

f. Quality Norms - The total number of errors for each work sample are 
converted to a three-point rating scale. The system also contains 
rate-of-learning norms for use during the training phase. 

g. Emphasis in Scoring - Time and quality are both given equal weight 
in the VIEWS. 

Observation of Clients 

a. Work Performance - Ten work performance factors (e.g., color discrim- 
ination, finger dexterity, work rhythm) are carefully defined. In 
addition to these definitions, specific definitions of the factors 
are made for each work sample. For example, in the Tile Sorting Work 
Sample, finger dexterity is assessed by "picking up tiles with fin- 
gers," in the Stamping Work Sample, it is "turning pages; picking up 
stamps." Each work sample has several factors listed that are to be 
observed. The accurate recording of behavioral observations is em- 
phasized. 
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b. Work Behaviors - Work oehaviors such as attendance and punctuality, 
response to training, and communication are clearly defined and ob- 
served during the course of the day. 

c. Recording System - Work behaviors and performance factors are written, 
as they are observed, on a client record form. Specific behaviors are 
reported on the forms; no rating system is used. 

d. Frequency of Observation - The VIEWS uses extensive observations. Ob- 
servation of defined work factors is required for each work sample. 
Work behavior observations are made daily. However, no established 
time or sample procedures are used for the work behaviors. 

Reporting 

a. Forms - The system uses four types of standardized forms: (1) a cli- 
ent record form for recording training observations, performance 
observations, behavioral observations, and errors (there is a sepa- 
rate page of this form for each work sample); (2) a daily observation 
form for summarizing work behaviors and performances; (3) a final re- 
port form, and (4) a profile sheet. 

b. Final Report Format - The VIEWS final report uses a standardized for- 
mat to present information on the following: general observations, 
interpersonal relations, training, worker characteristics, recommen- 
dations, and a profile sheet containing work sample results including 
the industrial time standards for the work samples. Recommendations 
are given for training techniques, Worker Trait Groups, and for other 
services that may be required. 

Utility 

a. Vocational Exploration - Since the tasks are work samples and not 
actual jobs and because almost no occupational information is pro- 
vided, the VIEWS is of little use in occupational exploration. 

b. Vocational Recommendations - Specific recommendations are made; these 
are related to the six Worker Skill Groups and from D.P.T. levels 
covered by the VIEWS. 

c. Counselor Utilization - The system and the final report are oriented 
toward the counselor. 

Training Required 

a. Training Required - Yes 

b. Training A. 1 able - Yes 

c. Duration - week in Philadelphia for new users. Under certain con- 
ditions, regional training is available. 
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d. Follow-up - One technical assistance visit is made to help with the 
establishment of the system and the management of standardized pro- 
cedures. 

11. Technical Considerations 

a. Norm Base - The VIEWS was renormed in 1979 on 452 mentally retarded 
persons (mean. IQ = 53) between the ages of 15 and 61. All data ar? 
reported only in terms of the 1-2-3 ratings. No means and standard 
deviations are given for the time and error scores for any of the 
work samples. MODAPT's predetermined time standard norms are also 
available. 

b. Reliability - No data presently available. 

c. Validity - No data presently available. 

12. Reviewer's Summary and Comments - The VIEWS attempts to evaluate the vo- 
cational potential ot mentally retarded adults for jobs in four D.P.T. 
levels. The system relates to job areas that are very common in the na- 
tional economy and more important to job areas where many retarded persons 
have found successful employment. The mst unique feature of the system 
is the attempt to separate learning from performance. The developers be- 
lieve that the client should first be thoroughly taught the task prior to 
performing it under timed conditions. The VIEWS also uses standardized 
behavior observations which are combined with time and quality scores to 
produce a well organized final report. The major problem with using the 
VIEWS by itself is the lack of occupational information. 

13. Address 

Vocational Research Institute 
Jewish Employment and Vocational Service 
1700 Sansom Street, 9th Floor 
Philadelphia, Pennsylvania 19103 

14. Cost 

$8,470.00 includes: work samples, manuals, forms, and tuition for train- 
ing one person in Philadelphia and one on-site visit by JEVS staff, 
excluding travel costs. Living expenses and transportation are not in- 
cluded in the price. 

15. References 

Backer, T. E., Client assessment: A manual for employment and training 
agencies . Los Angeles: Edward (ilaser 3 Associates, 1979. 

Cohen, B., VIEWS: New chance for the retarded. Performance, 1977, 27(9), 
13-17. " — 

Rosen, G. A., The Views Evaluation System. In A. Sax (Ed.), Innovations 
in vocational evaluation and work adjustment. Vocational Evaluatio n 
and Work Adjustment Bulletin , 1977, 10(2), 50-51^ 
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MODAPTS. . . industrial comparisons for VIEWS. The Sampler , 1977, 
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Vocational Interest Temperament and Aptitude System 

(VITAS) 



Development 

a. Sponsor - The VITAS was developed by the Philadelphia Jewish Employ- 
ment and Vocational Service under contract with the Manpower Ad- 
ministration. The system was originally designed for use within tra 
context of the U.S. Employment Service. JEVS is presently marketing 
the system to schools, rehabilitation centers and manpower programs. 

b. Target Group - According to the manual, "VITAS is designed for educa- 
tionally and/or culturally disadvantaged persons of both sexes. The 
system is not intended for individuals with more than a 12th grade 
education, the physically handicapped, or the mentally retarded." 
This reviewer, however, believes that the VITAS could be used with 
many physically handicapped persons and mildly retarded persons. 

c. Basis of the System - The VITAS is based on 16 Work Groups in the 
Guide for Occupational Exploration (GOE). These were selected be- 
cause of employment and/or training opportunities. 

Organization 

a. Name and Number of Work Samples - The 16 Work Groups are assessed by 
21 separate work samples listed below. (Note that several of the 
work samples assess for more than one work group.) 

(1) 02.04 - Laboratory Technology - #4 Collating Material Samples, 
#8 Nail & Screw Sorting (Part I); #8 Nail & Screw Sorting (Part 
II); and #20 Laboratory Assistant 

(2) 05.03 - Engineering Technology - #21 Drafting 

(3) 05.05 - Craft Technology - #11 Lock Assembly, #19 Spot Weldinq 
and #21 Drafting 

(4) 05.09 - Materials Control - #2 Packing Matchbooks, #3 Tile 
Sorting and Weighing, #5 Verifying Numbers, #8 Nail & Screw 
Sorting (Part II) 

(5) 05.10 - Crafts - #7 Budget Book Assembly, #8 Nail & Screw Sort- 
ing (Part I), #9 Pipe Assembly, #11 Lock Assembly 

(6) 05.12 - Elemental Work: Mechanical - #1 Nuts, Bolts & Washers 
Assemb>y, #2 Packing Matchbooks and #6 Pressing Linens 

(7) 06.01 - Production Technology - #8 Nail & Screw Sorting (Part I) 
#11 Lock Assembly, #12 Circuit Board Inspection, #19 Spot Welding, 
#20 Laboratory Assistant, and #21 Drafting 

(8) 06.02 - Production Work - #1 Nuts, Bolts & Washers Assembly, #6 
, n . ^ e " lp 9 L mens, #7 Budget B^ok Assembly, and #9 Pipe Assembly 

(9) 06.03 - Quality Control - #3 Tile Sorting and Weighing, #4 Col- 
lating Material Samples, #8 Nail & Screw Sorting (Part I), #12 
Circuit Board Inspection 

(10) 06.04 - Elemental Work: Industrial - #1 Nuts, Bolts & Washers 
Assembly, #2 Packing Matchbooks, and #6 Pressing Linens 
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(11) 07.02 - Mathematical Details - #8 Nail & Screw Sorting (Part II), 
i>i3 Calculating, #15 Bank Teller, and #17 Payroll Computation 

(12) 07.03 - Financial Detail - #8 Nail & Screw Sorting (Part II), 
#13 Calculating, #15 Bank Teller, #17 Payroll Computation, and 
#18 Census Interviewing 

(13) 07.04 - Oral Communications - #14 Message Taking and #18 Census 
Interviewing 

(14) 07.05 - Records Processing - #5 Verifying Numbers, #10 Filing by 
Letters, and #16 Proofreading 

(15) 07.06 - Clerical Machine Operation - #13 Calculating and #14 
Message Taking 

(16) 07.07 - Clerical Handling - #2 Packing Matchbooks, #3 Tile Sort- 
ing and Weighing, #10 Filing by Letters 

b. Grouping of Work Samples - Each work sample is independent in terms 
of administration and scoring. The results are combined and inter- 
perted as part of the above listed classifications. Work samples are 
also related to work groups. 

c. Manual - The manual contains the following information on each work 
sample: (1) inventory; (2) administration notes; (3) demonstration/ 
instructions, and (4) scoring procedures. A photograph is used for 
each work sample to insure proper setup. Although examples of re- 
port forms and definitions of the aptitude codes are given in the 
manual, there are no instructions on how to use these items. These 
are covered during training. 

Physical Aspects 

a. Packaging of the Work Samples - All work samples are packaged inde- 
pendently. Most of the work samples are packaged in plastic con- 
tainers that could easily be stored when not in use. The only heavy 
piece of equipment is the spot welder. 

b. Durability - The VITAS uses mostly basic tools and equipment that 
should be very durable. According to the developers, no problems 
have been reported. 

c. Expendable Supplies - The VITAS uses expendable items such as: paper 
and recording forms, string, and sheet metal. All supplies can be 
locally obtained. While there are no estimates, the cose per client 
administration is very low. 

d. Replacement - All replacement parts should be ordered from the de- 
veloper in order to maintain standardization. 

Work Evaluation Process 

a. Preliminary Screening - The manual does not mention that any prelim- 
inary screening is needed. 

b. Sequence of Work Sample Administration - The client usually begins 
with the least complex work sample (i.e., Nuts, Bolts and Washers 
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Assembly) and progresses to the more complex (i.e., Drafting). How- 
ever, the work samples can be given in any order. 

c. Client Involvement - The client is involved in the vocational process 
at several different times: (1) new clients are given an orientation 
session when first coming into the evaluation unit; (2) a group motiva- 
tional session at the end of the first day of evaluation; and (3) a 
feedback and interest interview after the work samples are administered 

d. Evaluation Setting - The VITAS manual stresses a realistic work setting 

e. Time to Complete the Entire System - According to the manual, "most 
clients can complete the work sample within three, five-hour days." 

Administration 

a. Procedures - A photograph showing the correct arrangement of parts is 
given to insure proper setup of each work sample. The instructions 
for the demonstration part of the work sample are brief and to the 
point; these include both oral and physical directions. For each 
work sample, a section called "Administration Notes" contains addi- 
tional items for setup and any special instructions that are to be 
fcl lowed by the evaluator during the client demonstration. 

b. Method of Instruction Giving - All client instructions are given or- 
ally and by demonstration. While the client is not required to read 
any administration instructions, reading and the use of mathematical 
skills are needed to successfully complete six of the work samples. 

c. Separation of Learning/Performance - Nine of the VITAS work samples 
do not have a separate practice period. After the evaluation in- 
structions and demonstrations are completed and any client questions 
answered, the client begins the task. No criteria are used to estab- 
lish that the client has learned the task. For all practical purposes, 
there is no separation of learning from performance. 

d. Providing Assistance to the Client - "When a client has a question or 
seeks assistance, the evaluator may repeat the necessary part of the 
instructions, including a re-demonstration, but should never do part 

of the task for the client. Only a minimum amount of assistance should 
be given so as to encourage the clients to do as much as they can on 
their own." Thus, when administering a work sample, the evaluator 
gives the instructions and demonstration and does not offer any addi- 
tional explanations unless requested. 

e. Repeating Work Samples - Re-administration of work samples is not rec- 
ommended. However, clients are urged to complete a task as best they 
can once they begin. 

Scoring and Norms 

a. Ti ing - The evaluator uses a time stamp machine to mark the starting 
time on a slip of paper; at the completion of the work sample, the 
client stamps his/her own slip. 
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b. Timing Interval - Timing begins when the evaluator completes the in- 
struction phase of the work sample and ends when the client completes 
the task. 

c. Time Norms - The minutes to completion are converted to 1-2-3 rating. 
No percentile or standard scores are used. 

d. Error Scoring - All work samples have carefully defined scoring pro- 
cedures in which errors are clearly described (e.g., "Number of books 
with string not tied in a bow.' 1 ) No random samples are used; the en- 
tire work sample is checked. The frequency of each type of error is 
recorded . 

e. Scoring Aids - Extensive use is mads of scoring aids such as templates, 
overlays, coding systems, and measuring instruments. 

f. Quality Norms - The errors are converted to a three-point quality 
rating. As with the time scores, no percentile or standard scores 
are used. 

g. Emphasis in Scoring - Both time and errors are given equal weight. 
Observation of Clients 

a. Work Performance - The VITAS manual describes nine work performance 
factors (i.e., aptitudes). The definitions of the factors (e.g., 
spatial, clerical perception, color discrimination) are taken from 
the DUT definitions. Specific definitions of each aptitude are re- 
lated to each appropriate work sample. Thus, in the Packing Match- 
book Work Sample, manual dexterity (M) is observed as "Transferring 
matchbooks from bin to tray; handling trays While in the Tile 
Sorting and Weighing Work Sample, M is observed by picking up boxes. 
Each work sample has from two to seven aptitudes that are to be ob- 
served. The close and accurate observation of client behaviors is 
stressed. 

b. Work Behaviors - General observations are to be made on attendance, 
punctuality, verbal ability, interpersonal behavior, and general 
worker characteristics. These are to be observed and recorded 
throughout the working day. 

c. Recording System - No rating or checklist system is used; specific 
behaviors for each work sample are recorded on a separate form for 
that work sample. 

d. Frequency of Observation - While the manual contains no specific in- 
structions as to when and how often to observe behavior, the system 
stresses the almost constant observation of client behavicr. 

Reporting 

a. Forms - The system uses five types of standardized forms: (1) a 
Work Sample Record Form for recording; aptitude and behavior obser- 
vations, types of errors, and time and error scores (there is a 
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separate page of this form for each work sample); (2) a general ob- 
servation form, (3) a final report form, (4) a Vocational Interest 
Interview Form, and (5) a profile sheet. 

b. Final Report Format - A four page final report form uses a standard- 
ized format to present information on the following: physical de- 
scription; attendance and punctuality, verbal ability, interpersonal 
behavior, skills, vocational recommendations by Work Groups, recom- 
mendations for supportive services, and profile of all work sample 
time and quality ratings. 

. Utility 

a. Vocational Exploration - The VITAS is of limited use for providing 
the client with occupational information, The nature of many of the 
tasks is abstract and no job information is provided during the in- 
struction period for each work sample. However, the manual states 
that occupational /vocational information should be used as a supple- 
ment to the VITAS. w 

b. Vocational Recommendations - Recommendations are made in two specif- 
ic areas: (1) the most feasible Work Groups for employment or 
training and (2) specific supportive services needed to obtain the 
employment goal. Apparently, recommendations within each work group 
are kept general - no specific jobs are suggested. 

c. Counselor Utilization - The final report is aimed at the counselor 
who needs to make fairly specific vocational decisions. 

Training in the System 

a. Training Required - Yes 

b. Training Available - Yes 

c. Duration - A one-week training session is held in Philadelphia. Re- 
gional training is available under certain conditions. 

d. Follow-up - One technical assistance visit is made to assist with the 
establishment of the system and the maintenance of standardized pro- 
cedures. 

Technical Considerations 

a. Norm Base - There are two norm groups for the VITAS. The 1980 norms 
contained results on a sample size of about 325 persons. The sample 
is clearly described— 63% female, white 63%, median age - 25 years 
and 35% 12th grade education. The 1981 secondary school norms have 
aoout 220 cases for each work sample. The mean age is 15.2 years, 
white - 78%, male - 68% and 67% learning disabled. Time and error 
scores are converted to a 1-2-3 rating system. No employed worker 
or predetermined time standard norms are given. 

b. Reliability - No data are available. 
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c. Validity - No data are available. The manual makes reference to "face" 
validity as a criterion and then confuses this with content validity. 



12. Reviewer's Summary and Comments - The VITAS System is the third work sample 
system developed by Philadelphia JEVS. Like the JEVS and VIEWS systems, 

it stresses the importance of careful and accurate behavior observations. 
The system also uses the work sample to work group approach that has served 
JEVS so well in the past. It must also be pointed out that many of the 
VITAS work samples are refinements and modifications of the original JEVS 
system. While the system could provide accurate assessment of CETA popu- 
lations in a relatively short period of time, it has two problems: (1) a 
lack of client occupational information, and (2) the failure to make any 
real distinction between learning and performance. The emphasis upon close 
client contact, careful observations, and the practical reporting format 
are the three major advantages of the system. 

13. Address 

Vocational Research Institute 
Jewish Employment and Vocational Service 
1700 Sansom Street, 9th Floor 
Philadelphia, Pennsylvania 19103 

14. Cost 

$8,886.00 includes work samples, manuals, forms, and tuition for training 
one person in Philadelphia and one on-site visit, excluding travel costs. 
Living expenses and transportation for the evaluator are not included in 
the price. 

15 . References 

Zimmerman, B., VITAS, In A. Sax (Ed.), Innovations in Vocational Evaluation 
and Work Adjustment. Vocational Evaluation and Work Adjustment Bulletin , 
1979, 12(1), 29-31. 

*Harris, J., Final report: VITAS work samples assessment as part of the Job 
Service demonstration project for out-of-school youth" ! (U.S. D0L con- 
tract #20428020), 1980. 
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Vocational Skills Assessment and Development Program 
(Brodhead-Garrett) 



Development 

a. Sponsor - The system was developed by the BroJ.head-Garrett Company of 
Cleveland, Ohio. 

b. Target Group - According to the manual, the system has been "designed 
for learners from ages 12 through adult with primary emphasis on the 
handicapped and the disadvantaged." The manual goes on to state that 
it can also be used with unemployed and underemployed persons. 

c. Basis of the System - The three manuals do not contain a discussion 
of the basis of the system. Apparently, the system is based upon the 
selection of vocational training programs and the providing of occu- 
pational information. No job classification using either the DOT or 
the U.S. Office of Education classifications is contained in the 
manuals. 



Organization 



a. Name and Number of Work Samples - The system contains three phases; 
only the first phase uses work samples. The second and third phases 
consist of specific tasks within a general vocational area. 

(1) Phase I - During this phase, the client is engaged in three types 
of activities. 

(a) Sorting - Six activities involving objects of different 
sizes and shapes as well as letters. 

(b) Assembly - Six activities in putting together pipes, col- 
lating, using a doorbell, nuts and bolts, and hand packaging. 

(c) Salvage - Six activities involving the disassembly of the 
items put together in the Assembly Component. 

(2) Phase II - The seven separate vocational components are intended 
to provide occupational information and to develop entry level 
skills: (1) basic tools, (2) sheltered employment, (3) building 
maintenance, (4) health, (5) agri-business, (6) clerical/sales, 
and (7) construction trades. Each of these seven component pro- 
grams is also composed of units. For example, the sheltered 
employment section includes the following: (1) collating, (2) 
engraving, (3) injection/rotation molding, (4) salvage/sorting, 
(5) packaging and (6) contracts/production. 

(3) Phase III - The seven separate program areas are designed to 
provide "basic job entry level skills for specific occupations": 
(1) health, (2) agri-business, (3) building maintenance, (4) 
clerical/sales, (5) automotive, (6) small engine, and (7) con- 
struction. As with Phase II, each Phase III program is composed 
of numerous subunits. For example, the small engine program 
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includes the following: (1) use of tools, (2) serivce and oper- 
ation, (3) disassembly of major parts, (4) assembly of major 
parts, and (5) diagnosing problems. 



b. Grouping of Work Samples - The program is divided into three phases 
as outlined above. It must be stressed that the Brodhead-Garrett is 
much more than a vocational evaluation system. Only Phase I can be 
considered as a work sample system as the term is used in vocational 
evaluation. Phases II and III are really generalized plans, goals, 
and curriculum for in-depth occupational exploration and skill train- 



c. Manual - The system has a separate manual for each phase. The Phase I 
manual contains instructions for administering and scoring as well as 
reporting forms. There are the evaluator's instructions; there are 
no detailed client instructions. The manual also lacks procedures 
for timing and scoring. The Phase II and III manuals contain general 
goals, methods, and outcomes. These manuals are well organized. 

Physical Aspects 

a. Packaging of Work Samples - Phase I work samples are contained in a 
large locking wooden cabinet. All work samples are stored within this 
cabinet. Each work sample is packaged individually. Phases II and 
III really cannot be judged because they are not work samples. 

b. Durability ~ The tools and materials appear to be very durable. 

c. Expendable Supplies - For Phase I, there is a minimal amount of ex- 
pendable supplies required. 

d. Replacement - While it is not stated in the Phase I manual, it is 
assumed that many parts can be replaced from local sources. 

Work Evaluation Process 

a. Preliminary Screening - Apparently no preliminary screening is re- 
quired. However, prior medical and psychological recomnendations are 
encouraged for use. 

b. Sequence of Work Sample Administration - According to the developer, 
the sequence is dependent upon the implementation of the system, i.e., 
the number of participants and scheduling. Although each phase is 
interrelated, any phase can be utilized without the other phases. 

Any appropriate assessment method can be used prior to the Phase II 
vocational exploration. 

c. Client Involvement - The manuals contain no information about giving 
client feedback. However, because the work samples must be admini- 
stered on an individual basis without formal instructions, it is 
expected that there would be a high degree of cl ient-eval uator con- 
tact. A Learner Comment Sheet is completed by the client; this 
general form includes likes and dislikes, duties performed, and 
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problems encountered. Phases II and III would assume a teacher- 
student relationship and here the amount of client involvement 
would depend upon the individual teacher. 



d. Evaluation Setting - Phase I implies either a classroom or an evalu- 
ation setting. Phases II and III are mostly educational in nature, 
although depending on the type of instruction, they could also be 
industrial . 

e. Time to Complete the Entire System - According to the manual, Phase I 
is "not to exceed six weeks." However, this reviewer estimates that 
the work samples could be given to most clients within the course of 
a full week. Phases II and III are training stages and each will 
last "at least six months" and until placement. Some of the training 
curriculum in Phase III could take over a year. 

5. Administration 

a. Procedures - Almost no procedures are specified for Phase I. Only 
the materials needed for each work sample are specified. 

b. Method of Instruction Giving - Instructions are given using a combi- 
nation of oral and demonstration techniques. 

c. Separation of Learning/Performance - According to the Phase I manual, 
"The instructor should demonstrate each activity and permit learner 
to practice before timing begins." However, no specific criteria are 
established and no mention of general criteria is made. 

d. Providing Assistance to the Client - No procedures are specified in 
the manuals. 

e. Repeating of Work Samples - The manual states "in case of excessive 
errors, demonstrate activity a second time and repeat exercise and 
timing ... If excessive errors are repeated, further study might 
be required." 

6. Scoring and Norms* 

a. Timing - The evaluator times the cl fent and records it on the rating 
sheet. 

b. Timing Interval - Timing begins after instructions and client prac- 
tice and ends when the task is completed. 

c. Time Norms - The time scores in minutes are reported on a three-point 
scale. 



Tor Phase I only; this section is not appropriate for Phases II and III. 
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d. Error Scoring - The Phase I manual does not specify how the tasks are 
to be scored for errors. Errors are not defined. 



e. Scoring Aids - No scoring aids are used. 

f . Quality Norms - A footnote at the bottom of each of three rating 
scales contains the only reference to error scoring and quality norms 
in the manual; it states "Deduct one point from score for each error 
over two—lowest score is 0. 1 ' There are no quality norms. 

g. Emphasis in Scoring - Time and quality are given equal weight. 
Observation of Clients 

a. Work Performance - No work performance factors are rated or discussed 
in the Phase I manual . 

b. Work Behaviors - Each manual provides two rating forms for work be- 
haviors; (1) the Adjustment Instrument contains 14 characteristics 
dealing mainly with personal and social adjustment (e.g., self-concept, 
responsibility, control of emotions), (2) the Work Habits Instrument 
contains 22 characteristics which focus on the performance of the tasks 
(e.g., attendance, use of work materials, ability to perform job). 
Both of these forms use a five-poinc rating scale; none of the factors 
are defined either in general terms or in behavioral terms. Specific 
work behaviors are not specified for any of the work samples. 

c. Recording System - Ratings on a five-point scale are used. The forms 
contain little space for comments. The ratings for each area are 
totaled for worker characteristics. 

d. Frequency of Observation - This is not specified in the Phase I manual. 
Reporting 

a. Forms - Time, errors, and a total score are recorded on a separate 
form for each of three Phase I components. As stated above, two forms 
are used for worker characteristics* 

b. Final Report Format - Manuals contain a four page report form contain- 
ing the following sections: dcnographic, initial information, voca- 
tional assessment, recommendations, academic history, and employment 
history. This form is to be updated as the person progresses through 
the system. A single page profile sheet is also given to plot the 
performance of tasks in all three components. The manuals contain no 
instructions for completing these forms. 

Utility 

a. Vocational Exploration - The system's Phase II and III offer the op- 
portunity for specific. occupational exploration in selected areas. 
In particular, Phase II is intended specifically for vocational ex- 
ploration. Because of the nature of the tasks and the lack of detailed 



instructions, Phase I appears to offer little in the way of career 
exploration. 

b. Vocational Recommendations - Each of the separate job areas for Phases 
II and III contain checklists of the major tasks covered in the areas. 
The overall level of proficiency is also rated. The usefulness of 
this data would depend on how it is used by client and teacher. 

c. Counselor Utilization - The Brodhead-Garrett manuals contain no men- 
tion of the rehabilitation counselor or a referral source. The system 
is designed to be used as an assessment and training device with the 
"end product" being a person ready for work. 

10 . Tr aining in the S ystem 

a. Training Required - No 

b. Training Available - Training is available. 

c. Duration - Two days to one week depending upon the number of phases 
and the components to be completed. 

d. Follow-up - This is provided on an as needed basis. 

11. Technical Considerations 

a. Norm Base - The manual does not contain uny norm data. 

b. Reliability - No reliability d;-ta are given in the manuals. 

c. Validity - No validity data are given in the manuals. 

12. Reviewer's Summary and Comments - The Brodhead-Garrett is a system that 
is intended to provide continuous service from initial assessment through 
training and evenutally job placement. In this aspect it is unique. 
Phase I is the only part of the system that can be considered as a work 
evaluation system as this term is usually used in vocational evaluation. 
Phase I lacks detailed evaluation and client instructions, norms, proper 
setup procedures, and scoring methods. In short, the manual for Phase I 
does not give the evaluator enough information to accurately use the sys- 
tem. In using Phase I, the evaluator must also ask how the content of 
assessment tasks is related to the specific training given in the other 
two phases. The success of Phases II and III depends on a large part 
upon the quality of instruction and the physical facilities. These two 
phases have the potential for being very useful for training clients in 
both basic skills and for some entry level positions. 

13. Address 

Brodhead-Garrett Company 
4560 East 71st Street 
Cleveland, Ohio 44105 
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14- Cost 

Phase I manual and equipment are approximately $5,950.00. Phase II and 
Phase III costs depend upon how many tools, equipment, and materials are 
presently available with the facility or school. If all Phase II and 
Phase III hardware and software were to be purchased > the cost would be 
chout $48,000. Software costs for each Phase II and Phase III component 
is C375.00 per manual . 

15. References 

None presently available. 
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Wide Range Employ ability Sample Test 



(WREST) 

Development 

a. Sponsor - The WREST was refined and is marketed by Jastak Associates. 

b. Target Group - The original work samples were aimed at supplementing 
the assessment of mentally retarded and physically handicapped per- 
sons in a sheltered workshop. According to the manual "it is par- 
ticularly appropriate for day activity centers, sheltered workshops, 
special education facilities, and other programs whose participants 
include the mentally retarded, cerebral palsied, and other severely 
physically, mentally, and socially handicapped." Its primary use is 
with persons for whom competitive employment of any kind is in doubt. 

c. Basis of the System - The WREST is based on a nroup of work samples 
originally developed at a sheltered workshop in Wilmington, Delaware 
for "referral of handicapped individuals who may be trained in basic 
work production skills." The work samples were used in conjunction 
with other techniques tc train and select persons for various areas 
of the workshop. 

Organization 

a. Name and Number of Work Samples - There are ten work samples. The 
first two of which each have two parts: 

(1) Folding includes (a) single fold and (b) double folding, gluing, 
labeling, and envelope stuffing, (2) Stapling includes (a) sta- 
pling accuracy and (b) collation and stapling, (3) Packaging, 
(4) Measuring, (5) Stringing, (6) Gluing, (7) Collating, (8) 
Color Matching, (9) Pattern Matching, and (10) Assembling. 

b. Grouping of Work Samples - Eac3i work sample is independent. 

c. Manual - The single manual contains the following major areas: (1) 
history, (2) theory, (3) general administration guidelines, (4) work 
sample instructions, (5) scoring, (6) technical considerations, and 
(7) case histories. The general administration section is highly 
detailed as well as useful. The instructions for each work sample 
are wt/il organized and easy to follow. A photograph is used for each 
work sample to insure proper layout. 

Physical Aspects 

a. Packaging of the Work Samples - The work samples, manuals, supplies, 
and scoring forms are all contained in a wood cabinet with drawers 
for each of the tasks. Thus, the entire system is placed in one small 
cabinet. 
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b. Durability - All work samples are made of heavy (mostly clear) plas- 
tic. The containers should be durable; however, the user should 
expect some wear of the pegs, tags, and colored pieces. 

c. Expendable Supplies - Besides forms, typing paper, stickers, and 
colored paper swatches are the most common expendable supplies. These 
are inexpensive and locally available. The developer also sells a re- 
supply kit. 

d. Replacement - Replacement parts can be ordered from the developer. 
Work Evaluation Process 

a. Preliminary Screening - No preliminary screening is required. 

b. Sequence of Work Sample Administration - The 10 work samples may be 
administered in any order. However, most evaluators "will find it 
more convenient to follow the designated order. " 

c. Client Involvement - The manual stresses that the client(s) should 
be told what the work samples involve and how the results will be 
used. The need for individualized attention is also mentioned. There 
is, however, no statement in the manual on providing feedback after 
specific work samples. 

d. Evaluation Setting - The evaluation setting would most likely be that 
of a formal testing situation. 

e. Time to Complete the Entire Syitem - Administration time for individ- 
ual clients is about one and a half hours; small groups of three to 
five persons take about two hours. 

Administration 

a. Procedures - For each work sample, the manual describes the purpose, 
and gives the materials, scoring information and instructions. A 
photograph is used to ensure proper layout, The WREST can be admini- 
stered to small groups of three to six persons. Duplicate sets of 
the WREST are necessary for group administration. 

b. Method of Instruction Giving - All instructions are oral and demon- 
stration; no reading is recuired. The manual warns that instructions 
must be closely followed: "any change from the manual may cause con- 
fusion, thus invalidating the norms of that test." 

c. Separation of Learning/Performance - Each work sample contains a prac- 
tice period prior to the start of timing. While there are criteria 
for most work samples (e.g... In Assembling, five practice items must 
be correct), the evaluator may use additional practice items ii nec- 
essary to make sure that the client understands the instructions. 
Thus, there is a separation of learning from performance. 

d. Providing Assistance to the Client - The manual clearly states that 
"once the formal testing has b^gun, no help can be given, but all 
possible assistance should be given during the instruction and prac- 
tice preceding the formal testing." 
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e. Repeating Work Samples - Readministration of work samples is empha- 
sized for upgrading. Evaluators are encouraged to keep accurate 
records of all readministration. 

Scoring and Norms 

a. Timing - The evaluator times the client(s) using a stopwatch or other 
timing device. 

b. Timing Interval - Timing is started after the client(s) understand(s) 
the task and continues for a set period of time, which is different 
for each task. The time needed to complete each work sample is re- 
corded in minutes and seconds. 

c. Time Norms - The number of minutes and seconds recorded are compared 
to scaled scores ranging from 0 through 19. The scaled scores can 
also be compared to standard scores. 

d. Error Scorinq - All completed parts are checked against the clearly 
defined scoring criteria given in the manual. 

e. Scoring Aids - No use is made of scoring aids. 

f. Quality Norms - The errors for all ten work samples are added together 
and the total compared v o norm tables. 

g. Emphasis in Scoring - The time results are emphasized. 
Observation of Clients 

a. Work Performance - The manual states that the evaluator should be 
familiar with the dexterity and perceptual aptitudes as defined in 
the DOT. However, no instructions are given for making, recording, 
and using these observations. 

b. Work Behaviors - Ten general work behaviors (e.g., appearance, per- 
severance, organization of work, and safety practices) are defined in 
the manual. These are not defined in behavioral terms. There is a 
space on the Summary Profile for rating each behavior category. 

c. Recording System - Each behavior is rated on a scale from one to 18. 
Verbal descriptions of from "very poor 11 to "very good" are used in 
conjunction with the numbers. There is no explanation of how these 
ratings are to be obtained; there is almost no room on the form for 
recording actual observations. 

d. Frequency of Observations - This is not specified in the manual. 
Reporting 

a. Forms - A single, two page form is used to record all raw scores and 
to report the converted scores, as well as the "behavior" observations. 
The second page of the form contains space for a work history and 
"summary and recommendations." 



b. Final Report Format - A variety of reporting formats are illustrated 
in the manual. These show examples of final reports which incorpo- 
rate a wide variety of data from other sources. The WREST was not 
intended to be used independently of other methods of assessment. 

9. Utility 

a. Vocational Exploration - The very simple nature of most of the work 
samples makes the WREST of little use in job exploration for a normal 
population. 

b. Vocational Recommendations - The manual contains no information on 
the making of vocational recommendations from the work sample results, 

c. Counselor Utilization - The manual contains no information on use of 
WREST results for the counselor. 

10. Training in the System 

a. Training Required - No training is required prior to purchase or use. 

b. Training Available - No formal training is available. 

c. Duration - Not applicable. 

d. Follow-up - Not applicable. 

11. Technical Considerations 

a. Norm Base - Time and quality norms are available on three major groups: 
(1) general population, (2) sheltered workshop employees, and (3) com- 
petitively employed workers. The general population group is further 
broken down into six age groups and by sex. In the workshop and in- 
dustrial groups, ages and sex were combined when it was discovered that 
there was little significant differences within these general groups. 
Norm groups are well defined and sample sizes range from 200 for in- 
dividual groups to 4000 for large groups. All samples were collected 
in the State of Delaware. 

b. Reliability - Test-retest reliability coefficients for time and error 
scores on the ten work samples were calculated using 428 employed work- 
ers over a three month period. All correlations were in the .90' s. 

A second study on a very small sample (N=15) repeated the WREST three 
times over a period of a few weeks; the correlations were in the high 
.80's and low .90's. These results are a strong indication of the 
test-retest reliability of the WREST. 

c. Validity - Validity is based on two correlations between supervisor's 
ratings and time and error standard scores for 428 employed workers. 
The WREST correlated .86 (time) and .92 (quality) with the ratings. 
These correlations are extremely high; so high in fact that the manual 
advises that "extreme caution must be used in regarding such studies 
as the above as valid measures of test validity." Nevertheless, the 
results are encouraging. 
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Reviewer's Summary and Comments - The WREST consists of ten, short, low- 
level tasks apparently designed to assess mainly the manipulation and 
dexterity abilities of the client. Although it is not stated in the man- 
ual, the WREST seems most useful in assessing new clients for assignment 
to suitable work projects within a sheltered workshop. The emphasis 
upon repeating the work samples many times should provide an evaluation 
of the client's ability to improve his performance under repeated prac- 
tice conditions. The major problems of the system center around the lack 
of systematic behavior observations, failure to relate results to the 
competitive job market, and the apparent lack of a useable final report 
for the referring counselor or agency. Finally, the WREST has an adequate 
norm base, good estimates of test-retest validity, and an attempt at es- 
tablishing concurrent validity. In a field that is all too often 
characterized by poor technical development, the WREST can serve as a 
good example. 



Address 



Jastak Associates, Inc. 
1526 Gilpin Avenue 
Wilmington, Delaware 19806 



Cost 



Work Sample Kit 

$1,395.00 
1,095.00 
25.30 
115.00 
9.25 
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Work Skill Development Package 
(WSD) 



1. Development 

a. Sponsor - The WSD package was developed by the Attainment Co, 

b. Target Group - The package is designed to develop work skills in 
severely "mentally disabled persons." 

c. Basis of the System - The WSD reflects three basic prevocational 
skills: (1) ability to discriminate between objects, (2) manipulate 
objects, and (3) apply basic concepts. While the WSD is intended to 
be mainly a training package for prevocational skills, it can be 
used as a vocational evaluation system. 

2. Organization 

a. Name and Number of Work Samples - The system consists of 20* work 
samples arranged in three groups: 

(1) Discrimination Tasks - Three Item Sort, Basic Size Discrimination, 
Tactile Discrimination, Cue Variable, Subtle Color, Subtle Size 
Discrimination, Six Item Sort, Rubber Parts Sort, and Twenty-four 
Item Sort. 

(2) Assembly Tasks - Tube Assembly/Disassembly; Paint Brush Assembly/ 
Disassembly; Coupling Assembly/Disassembly; Container Assembly/ 
Disassembly; Connector Assembly/Disassembly; Pen Assembly/Disas- 
sembly and Shelf Assembly/Disassembly. 

(3) Packaging Tasks - Color Match Collating/Collating Disassembly; 
Snap Box Packaging/Disassembly; Small Parts Packaging/Disassembly; 
Plate Weighing and Bagging/Plate Disassembly. 

b. Grouping of Work Samples - The work sample tasks are grouped in two 
different ways: First, by function as listed above. Second, by 
difficulty. When classified by difficulty, there are four series of 
five tasks each; each series contains tasks taken from the discrimi- 
nation, assembly, and packaging. 

c. Manual - All directions are contained in a single loose-leaf binder. 
The manual is organized by work sample function. The following in- 
structions are given for each work sample: materials, procedure, 
quality criteria, norms, and any notes of explanation. There are 
also illustrations showing the proper set-up of the task. The ad- 
ministration instructions are purposely vague; the trainer is to 
vary instructions according to the client's need. 



*Assembly/Disassembly and Package/Disassembly units are counted as one work 
sample. 
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Physical Aspects 



a- Packaging of the Work Samples - Each work sample is individually pack- 
aged and contained in one or more bins. No parts are shared between 
work samples. 

b. Durability - The testing materials are made from durable metal, wood, 
plastic, and rubber. The system appears durable. 

c. Expendable Supplies - Aside from forms and plastic bags for heat 
sealing, there are no expendable supplies. 

d. Replacement - Some replacement parts and supplies are provided as ex- 
tras with the initial package. Other supplies may be ordered from 
the developer. 

Work Evaluation Process 

a. Preliminary Screening - The manual contains no information on prelim- 
inary screening. 

b. Sequence of Work Sample Administration - The tasks are administered 
according to the four difficulty categories. The client repeats each 
task until he/she meets the existing criteria; then he/she goes on to 
the next higher category. 

c. Client Involvement - The manual contains no information about client 
involvement in the training process and no formal feedback procedures 
are given. 

d. Evaluation Setting - This is not specified. However, given the nature 
of the tasks, they would fit into both a school setting and a work 
activity center ^evel evaluation or work setting. 

e. Time to Complete the Entire System - While this is not specified in 
the manual, this aspect is not relevant because the system stresses 
basic training. 

Administration 

a. Procedures - The materials, procedures, illustrations, etc., are all 
easy to understand. 

b. Method of Instruction Giving - The evaluator is !, to model the correct 
procedure for the worker as outlined in the Procedure Section 1 ' for 
each task. Verbal instructions are also used. The evaluator is 
urged to be flexible and modify instructions according to the needs 
of the individual client. 

c. Separation of Learning/Performance - While the manual allows for a 
short practice time, prior to timing, no specific instructions on 
separation of learning from performance are given. 

d. Providing Assistance to the Client - The evaluator has the option of 
providing extra instruction. t rw > 
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e. Repeating Work Samples - Tasks may be repeated as training devices 
and as a way of reaching the exit criteria for each of the four 
Series of Tasks. The repetition of the tasks is partly dependent 
upon meeting the success criteria for each task. 

Scoring and Norms 

a. Timing - Not specified. 

b. Timing Interval - Not specified; the manual seems to imply that the 
evaluator begins to time the client after the practice session and 
stops when the last object has been sorted, assembled, disassembled, 
or packaged. 

c. Time Norms - MTM norms are provided for the assembly/disassembly and 
the packaging tasks. There are no time norms for the discrimination 
tasks. 

d. Error Scoring - Each piece is checked against a well-defined criteria. 

e. Scoring Aides - No scoring aides are used. 

f. Quality Norms - There are no quality norms; a percentage of errors is 
recorded. 

g. Emphasis in Scoring - Both time and accuracy are equally important. 
Observation of Clients 

a. Work Performance - No specific performance factors are mentioned in 
the manual . 

b. Work Behaviors - No work behaviors are defined and the manual does 
not contain work behaviors to be observed. This is inconsistent with 
literature stating that one application of these tasks is "The devel- 
opment of appropriate work behavior and acceptable work habits. 11 

c. Recording System - No recording system is used. There is a space for 
comments on ths Work Performance Data Sheet. 

d. Frequency of Observation - This is not specified. 
Reporting 

a. Forms - The only form used is the Work Performance Data Sheet which 
contains columns for the specific task, time, MTM norms, baseline, 
goal , and comments. 

b. Final Report Format - Because the WSD Package is a training device 
and not a work sample, no final report format is available. 




9. Utility 



a. Vocational Exploration "1 According to the developers, the system 

. is used as a "prevocational training and 

b. Vocational Recommendations V assessment program" to teach the basic 

[ skills discrimination, assembly, and 

c. Counselor Utilization J packaging. 

3. Training in the System 

a. Training Required - Training is required prior to use. 

b. Training Available - In-service training is provided by the developer 
when the system is installed. 

c. Duration - The session lasts one day. 

d. Follow-up - Training and follow-up are included in the price of the 
package. 

Technical Considerations 

a. Norm Base - MTM norms are available on all the assembly and packaging 
jobs. 

b. Reliability - No studies are reported in the manual. 

c. Validity - No studies are reported in the manual. 
. Reviewer's Summary and Comments 

The introduction to the Work Skill Development Package contains the fol- 
lowing statement ". . . the WSD Package is not a work evaluation system. 
It is not intended to evaluate or predict vocational readiness through a 
single administration of tasks." The system is a prevocational training 
program intended to be part of a skill and "behavior 11 acquisition pro- 
cess. The WSD's major purpose is as a training device for mentally 
retarded, mentally ill, and developmental ly disabled persons. The log- 
ical progression in complexity and working with different materials makes 
the system easy to use with moderate and severely mentally retarded per- 
sons. However, if the WSD Package is to be used as a training device, 
it needs to have an explanation on how to record, plot, and use the re- 
sults from several administrations. 

Address 

Attainment Co. 
P.O. Box 103 

Oregon, Wisconsin 53575 
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14. Cost 

Current price is $2,995-00; this includes shipping, delivery, in-service, 
and replacement parts. 

15. References 
None 
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